HDM Introduces Zero-or-T Thinking to Cut Inference Latency and Token Costs
·2026.11.16 19:30
Key point
The hybrid decision model uses a lightweight router to switch between implicit and autoregressive reasoning, reducing median latency to 0.179 s on JevBench.
Details
- Introduces a "zero-or-T thinking" paradigm where a single model either answers from implicit thinking in one forward pass or performs full autoregressive reasoning at a cost of T tokens.
- On JevBench, HDM-4B scores 0.857 compared to Jev's 0.851 (+0.7%) while reducing median latency from 0.652 s to 0.179 s (-72.5%).
- On the agentic venue MiniWoB, HDM-27B cuts token cost by 80.1%.
- Utilizes a lightweight self-routing block that estimates the utility of slow-mode reasoning against a per-request price lambda to decide when to engage full reasoning.
- Proposes Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe for training the routing block from episode-level returns in tasks without per-decision labels.
- Evaluated on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77, and MiniWoB.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.