Ring-Zero: Emergent Reasoning via Zero RL Scaled to 1 Trillion Parameters

Scaling Zero RL to 1 Trillion Parameters

This paper reports the first case of applying Zero Reinforcement Learning (Zero RL), which uses verifiable rewards without human annotation, to a large language model at the 1-trillion-parameter scale. Prior work was limited to small models due to computational constraints, but this study explored the training dynamics and emergent abilities at large scale.

Key Technical Improvements

Naive scaling suffered from low readability, token duplication, and a lack of adaptive reasoning depth. The research team presented a stable and efficient training pipeline that includes:

• Truncated importance sampling
• Training-inference ratio correction
• Mixed-precision control

Key Findings

1. Benefits of Scale: Scaling to 1 trillion parameters significantly improves sample efficiency and the performance ceiling
2. Training Phases: Sequential progression from an initial discovery phase to a refinement phase
3. Automatic Emergent Abilities: Advanced cognitive behaviors—including anthropomorphization, structured formatting, self-verification, parallel reasoning, and context anxiety—automatically emerge without hand-crafted heuristics

Evaluation Results

Ring-2.5-1T-Zero achieved competitive performance across 7 math benchmarks. Beyond final answer accuracy, the team proposed a structured evaluation framework that assesses chain-of-thought (CoT) quality along three dimensions—comprehensibility, reproducibility, and efficiency—and the model showed a clear advantage in generating structured and concise reasoning traces.