Adaptive Thinking: LLMs Judge When to Think in Latent Space

Test-time compute for LLMs has shown that allowing long chain-of-thought (CoT) reasoning before an answer can improve performance, but it remained unclear how much reasoning budget should be spent on which query. Using self-consistency—the degree of agreement across multiple reasoning paths—as a proxy for reasoning necessity provides a signal that questions with lower agreement need longer thinking.

Sonata (Self-Consistency-Guided Adapter for Thinking Allocation) trains a lightweight adapter on an offline calibration dataset so that it predicts self-consistency by looking only at the last-layer hidden representations from the query's prefill stage. This prediction is used to set the reasoning budget before actual reasoning takes place, adding almost no overhead during inference. Once trained, the adapter transfers across various tasks, and it operates independently of existing CoT compression, so combining the two can yield additional efficiency.

• Verified on Qwen3-8B, GPT-OSS-120B, Qwen3-235B-A22B, and Intern-S1-mini.
• On AIME24, AIME25, GSM8K, MATH500, and GPQA, it reduced reasoning tokens by 20%~80% while maintaining the same accuracy.
• At the same token cost, it also confirmed accuracy improvements of up to 5%.