AI BriefingHead-to-head battle of small LLMs
Key point
mii-llm released the 0.4B multilingual models Zagreus/Nesso, trained on 64×A100.
Details
mii-llm released technical reports for the Zagreus and Nesso model families.
Both model families were trained from scratch at 0.4B parameters scale, and were designed with edge deployment, multilingual capability, and European language support as goals.
Pretraining was conducted with a bilingual composition of English + target language, with target languages being Italian, Spanish, French, and Portuguese.
The released models are as follows.
- Zagreus-0.4B-ita: English/Italian base
- Zagreus-0.4B-spa: English/Spanish base
- Zagreus-0.4B-fra: English/French base
- Zagreus-0.4B-por: English/Portuguese base
- Nesso-0.4B-instruct: post-trained conversational version
- Nesso-0.4B-agentic: post-trained version for structured/agentic tasks
- Open-Zagreus-0.4B: a fully open variant built with open data and open recipes
The training stack consisted of Datatrove (tokenizing), Hugging Face Nanotron (pretraining), Axolotl (post-training), and Slurm (multi-node orchestration), with hardware of 64 NVIDIA A100 GPU and a training token scale of about 1 trillion tokens.
As a design choice, a dense 0.4B structure was chosen instead of MoE. The report explains that in the sub-1B range, stability and resource utilization may matter more than sparse efficiency.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.