PickKolibri-1: 78B MoE Model with 3.46B Active Parameters per Token for Low-Cost Inference

Kolibri-1, released by Aleph Alpha, adopts a Mixture-of-Experts architecture with a total of 78B parameters, yet only 3.46B parameters are activated per token. This significantly reduces inference costs while maintaining the performance of large-scale models. It maximizes memory efficiency through FP8 quantized weights and is optimized for vLLM-based deployment.

Bilingual support specialized in German and English is a key feature. It was pretrained on a dataset of approximately 20 trillion tokens, with about 24% consisting of German data. It delivers high accuracy across various tasks, including complex multi-step reasoning, coding, RAG system integration, and agentic tool calling.

The default context length is 262,144 tokens, extensible up to 1,048,576 tokens using sliding window attention. It features built-in explicit reasoning mode and tool calling capabilities, making it suitable for automating complex workflows. It is licensed under Apache 2.0 for free commercial use and complies with the EU GPAI Code of Practice.