Gair NLP announces 801-30B-A3B-Base, a model trained via recursive self-improvement in pre-training

Details

Gair NLP reports 801-30B-A3B-Base, the first frontier pre-training run to apply recursive self-improvement (RSI) to the entire production system rather than just post-training.
The 30B-A3B MoE model was trained from scratch on 11.8T tokens over three months by a newly formed team.
As a base model, 801-30B-A3B outperforms Qwen3-30B-A3B and Nemotron-3.5-Lightning-30B-A3B, trailing only the larger Qwen3.5-35B-A3B.
With lightweight SFT (6B tokens, no RL), the model achieves the highest average on four agentic benchmarks against fully post-trained models, including Qwen3.6-35B-A3B.
The team decomposes RSI into smaller loops to address the high cost and non-repeatability of single pre-training runs.
The base model and lightweight SFT data are scheduled to be open-sourced, with the model release coming soon.

Key point