New Technique Compresses LLM Reasoning Tokens by 2-3x

Through compression training using Self-distilled reasoning traces on the Qwen3.5-4B and Gemma-4-12B models, it was confirmed that these models use 2-3x fewer tokens compared to existing models while maintaining or even improving performance.

The key research findings are as follows:

• Section-aware Compression: Rather than compressing the entire reasoning process, it is more effective to preserve the Compute and Verification segments while compressing only the narrative/transition phrases between them. This approach showed a +0.15 performance improvement over uncompressed SFT on the GSM8K benchmark.
• Termination Anchor: The model recognizes the computation segment not merely as a working memory device, but as an Anchor that terminates reasoning.
• Prompt Dependency: Compression-trained models respond to system prompts like 'Step by step' as an efficiency trigger, and the most effective approach is to train without the prompt and then use the prompt at serving time.

The research findings, model, and code have all been made publicly available.