Claude Opus 4.7 Comparison

In Claude Code CLI 2.1.117, Opus 4.5, 4.6, 4.7, and Sonnet 4.6 were compared across the 5 levels low, medium, high, xhigh, and max. 10 fixed prompts were run against each combination, with total cost around $18.

• session-metrics read Claude Code's JSONL transcripts to reconstruct tokens and pricing.
• Following 2.1.116+, which Anthropic's post-mortem mentioned as the fix for the quality regression, this test was conducted on 2.1.117.
• effort was an output token dial, not input tokens. Input tokens per prompt were nearly identical at about 6.

The pattern across models split into roughly three groups.

• Opus 4.6, 4.5, and Sonnet 4.6 started thinking on 90-100% of turns even at low.
• Opus 4.7 had an 18% thinking rate at low but rose to 93% at max.
• Sonnet 4.6 kept tool calls active at every rung, and since it was cheaper than Opus at the same rung, its dollar efficiency per token was 2-5x higher. The price table itself was 60% of Opus's.

Cache state significantly shook up the results. Even with the same model and same effort, whether the 1-hour cache was warm or cold caused up to a 1.88x difference — for example, Opus 4.7 [1m] xhigh was $1.12 (warm) vs. $1.78 (cold), and Sonnet 4.6 medium was $0.78 vs. $1.46. In the warm state, the efficiency gap between Sonnet 4.6 medium and high was only about 5%.

By pure efficiency, Sonnet 4.6 high achieved 15,516 tok/$, while Opus 4.7 high achieved 3,015 tok/$. Conversely, Opus 4.7 max produced up to 6.5x more output tokens than low, behaving almost like a separate mode, whereas Opus 4.5's increase at the same max was only 2.1x.

With IFEval, the sample size was only 9, so the McNemar p-value stayed at 0.25-0.5, a range in which a significant difference cannot be confidently claimed.