AI BriefingSoft-MTP paper identifies flaw in standard Multi-Token Prediction and proposes a one-line fix
·2026.10.30 19:59
Key point
The method removes a noisy gradient component in standard MTP, improving perplexity and speculative decoding speed from 167M to 1.2B parameters.
Details
- Standard Multi-Token Prediction (MTP) training contains a residual gradient component that interferes with learning, an effect that worsens with model scale.
- Soft-MTP uses a KL gradient-recovery trick to isolate the useful self-distillation signal, implemented as a one-line code change with no added parameters or measurable overhead.
- Across scales from 167M to 1.2B parameters, Soft-MTP achieves lower next-token perplexity, faster speculative decoding, and higher zero-shot accuracy compared to standard MTP.
- The method supports both DeepSeek- and Meta-style MTP architectures and matches or exceeds direct KL divergence minimization on most metrics.
- Lower MTP perplexity is observed from 689M parameters onward, with stronger math and code reasoning performance after fine-tuning at 1.2B parameters.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Decagon's AutoResearch Agent Automates Inference Stack Optimization, Achieving Up to 2.5x Throughput GainsDeepSeek Open-Sources DeepInfer Framework for V4.1 Flash Inference