SantaCoder

SantaCoder is the flagship 1.1B-parameter model of the BigCode open-science collaboration (ServiceNow/Hugging Face), described in the December 2022 tech report 'SantaCoder: Don't Reach for the Stars!' (arXiv:2301.03988). Trained on the Java, JavaScript, and Python subsets of The Stack v1.1 — a 6.4TB dataset of permissively licensed code with PII redaction and an opt-out mechanism — it used multi-query attention and fill-in-the-middle training for efficient code infilling.

Its headline result was efficiency: despite being a fraction of the size of rivals, it outperformed the open-source InCoder-6.7B and CodeGen-Multi-2.7B on MultiPL-E generation and infilling benchmarks. All models were released under an OpenRAIL license — open weights with use-case restrictions, not OSI-open — and SantaCoder directly paved the way for the 15.5B StarCoder that followed.