AI BriefingPerplexity Proposes JOLT: Joint On-Policy Learning and Teaching for Efficient RL Post-Training
·2026.10.21 09:00
Key point
JOLT jointly trains a single model as both teacher and student to provide dense, aligned guidance, achieving higher accuracy and significantly better sample efficiency than GRPO on math, coding, and agentic benchmarks.
Details
- JOLT (Joint On-policy Learning and Teaching) uses a single model in two roles: a privileged teacher with access to verified solutions and a student without such access, training both jointly to align teacher feedback with the student's learning capacity.
- JOLT+ achieves the highest mean score across all evaluated benchmarks, outperforming the strongest baseline (GRPO) by 1.5pp on GSM8K, 4.2pp on MATH, 3.0pp on LCBv6, and 9.9pp on AppWorld state-test pass rate.
- On Terminal-Bench 2.1, JOLT+ leads with the highest pass@k at every reported k, ranging from 10.96% at pass@1 to 34.83% at pass@32.
- JOLT demonstrates superior sample efficiency on MATH, requiring 6.5x fewer processed training tokens (1.30M vs 8.40M) and 13x fewer completion tokens than GRPO to reach a matched accuracy of 63.44%.
- The method addresses the sparsity of outcome-reward RL by using On-Policy Distillation (OPD) signals, where the teacher's objective pushes its feedback to move the student in the same direction as RL on the student's own reward.
- Experiments were conducted using base models including DeepSeek-V2-Lite, Qwen3-0.6B, Qwen3-8B, Qwen3-4B-Instruct, and Qwen3-30B-A3B-Instruct across mathematical reasoning, coding, and long-horizon terminal tasks.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Stanford Researchers Propose EXPO-FT for Universal Post-Training in RoboticsFlapping Airplanes Valuation Jump to $5B Raises Questions on AI Timeline and Treaty Enforcement