Agentic Test-Time Scaling GitHub Repository

AutoTTS automatically discovers Test-Time Scaling controllers through code editing alone in an offline replay environment, instead of having humans write heuristics by hand. Humans only define the state, action, feedback, and objective, then build an environment that replays cached reasoning trajectories, while a coding agent iteratively revises the policy code within it.

The environment pre-collects multiple reasoning traces for each query, decomposes them into fixed-length segments and probe responses, and stores them in a replay store. Candidate policies are then evaluated using only observe/step, with zero LLM calls during the discovery phase. Every hyperparameter inside the policy is determined by a single scalar β, so the search is designed to essentially converge into a β sweep.

Optimization was carried out on the AIME24 replay configuration, and evaluation was performed on the AIME25 and HMMT25 held-out benchmarks. The target backbones are four scales of Qwen3, and the discovered controller showed a better accuracy-token trade-off than handcrafted baselines such as SC@64, ASC, ESC, and Parallel-Probe.

• At β=0.5, token usage was reduced by about 69.5% compared to SC@64, while matching the held-out average accuracy across all four backbone scales.
• A single discovery run cost about $39.9 and took 160 minutes, and the evaluation phase used replay only, with 0 LLM calls.
• At β=1.0, it achieved the highest accuracy surpassing handcrafted baselines in 5 out of the 8 comparison cells in the project's table.

The discovered policy is the Confidence Momentum Controller (CMC). It looks at both the EMA and the trend of pool confidence together, so it doesn't react to momentary spikes and stop. When confidence stagnates, it widens branches, and it allocates more depth to branches with good alignment. Conversely, it conservatively discards branches that consistently diverge, adjusting width and depth together.