Hugging Face Releases Code LLM StarCoder

StarCoder and StarCoderBase are code-specialized large language models (Code LLMs) trained on permissively licensed data from GitHub.

StarCoderBase has approximately 15B parameters and was trained on 1 trillion tokens, while StarCoder was further fine-tuned on 35B additional Python tokens based on StarCoderBase to maximize performance. It supports a context length of over 8,000, making it advantageous for handling long code inputs, and can perform code completion, modification, explanation, and serve as a technical assistant.

Key Performance and Features:

• Benchmarks: On the HumanEval test, it achieved 40.8% with prompt optimization, reaching state-of-the-art (SOTA) level among open-source models. This figure matches or exceeds OpenAI's code-cushman-001.
• Multilingual Support: It demonstrated strong performance across various programming languages through the MultiPL-E benchmark.
• Safety and Licensing: A PII removal pipeline and attribution tracing tools were introduced, and the OpenRAIL license was applied to enhance usability for enterprises.