Vals AI Finds 4 of 7 Frontier Models Keep 80% of Score at Lowest Reasoning Setting for Half Cost or Less

• Four of seven tested frontier models maintained at least 80% of their Vals Index score at their lowest reasoning setting for half the cost or less.
• Claude Opus 5.5 and Grok 4.7 achieved the highest efficiency, keeping 80% and 86% of their scores at only 16% of the leaderboard cost.
• In 22 of 56 model-benchmark combinations, a lower reasoning setting scored as high as the leaderboard setting at a lower cost, with no lower setting scoring significantly higher.
• Selecting the highest-scoring setting for each individual task with perfect hindsight adds only 6.0 points to the median score while increasing cost to 64% of the leaderboard setting, compared to 29% for the lowest setting everywhere.

Benchmark Variance

• Legal Research showed the largest performance gain from increased reasoning, with a median increase of 19.2 points from lowest to highest setting.
• EMB 1.1, HLAB 1.1, and Finance Agent 2.1 showed minimal gains, each increasing by less than 6 points.

Methodology and Limitations

• Tests used the top model from seven AI labs on eight Vals Index benchmarks, running each at lowest, middle, and leaderboard settings.
• Scores are based on a fixed subset of about half the tasks per benchmark, making results noisier than full-benchmark runs.
• The 6.0-point potential gain is an upper bound because it assumes a system can pick the best setting after seeing results, which is not possible in real-time deployment.