StarCoder2
Open code LLM from BigCode
StarCoder2 is a family of open-weight large language models specialized for code generation and understanding, developed by the BigCode project (a collaboration between Hugging Face and ServiceNow, with hardware support from NVIDIA),…
Definition
StarCoder2 is a family of open-weight large language models specialized for code generation and understanding, developed by the BigCode project (a collaboration between Hugging Face and ServiceNow, with hardware support from NVIDIA), trained on permissively licensed source code across hundreds of programming languages.
Overview
StarCoder2 is the successor to the original StarCoder model and was trained on The Stack v2, a large corpus of source code drawn from permissively licensed repositories, software heritage archives, and related technical content such as GitHub issues, pull requests, and Jupyter notebooks. The family was released in three sizes — 3B, 7B, and 15B parameters — giving developers a range of options to trade off inference cost against code-generation quality. A defining characteristic of the BigCode project, and StarCoder2 specifically, is its emphasis on training data transparency and licensing: the project publishes tooling that lets developers check whether their own code appears in the training set, and restricts training data to code under permissive open-source licenses, addressing concerns about training on copyrighted or restrictively licensed code that surround some other code models. This transparency-first approach distinguishes StarCoder2 from code models trained on less disclosed data mixtures. StarCoder2 supports a large context window suited to whole-file and multi-file code understanding, and it performs well across a broad set of programming languages rather than specializing narrowly in one. It is commonly used as a base model for code completion tools, fine-tuned coding assistants, and research into code-specific model behavior, and its open weights and permissive license make it a popular choice for organizations that need to self-host a code model or that have licensing concerns about training data provenance.
Key Concepts
- Released in 3B, 7B, and 15B parameter sizes
- Trained on The Stack v2, a curated permissively licensed code corpus
- Developed by the BigCode collaboration (Hugging Face, ServiceNow, NVIDIA)
- Emphasizes training data transparency and license compliance
- Supports a large context window for whole-file and multi-file code tasks
- Broad multi-language programming support
- Open-weight release suitable for self-hosting and fine-tuning
- Widely used as a base model for code completion and coding assistants