AI BriefingOpus 4.7 vs Old Opus 4.6 vs New Opus 4.6 on 28 Zod tasks
Key point
On 28 Zod tasks, Opus 4.7 had the same pass rate as 4.6 but was faster and more efficient.
Details
Based on Zod's 28 historical tasks, Opus 4.6 (March 19), new Opus 4.6 (April 16), and Opus 4.7 (April 16) were rerun under identical conditions.
- All three versions had the same test pass rate of 12/28 pass rate.
- However, 4.7 showed more restrained coding overall, showing a better direction in the craft and discipline of the patches it wrote.
- In cost and speed, 4.7 was more efficient than March 4.6.
- Cost per task: $8.11 vs $8.93
- Total tokens: 44.0M vs 49.1M
- Total run time: 1h 30m vs 1h 36m
- The newly rerun 4.6 was the cheapest, but it was evaluated as taking longer and producing looser, less equivalent patches.
The author ran each Opus snapshot on Claude Code based on the same commit, and scored not just simple pass/fail but also equivalence, code-review pass, footprint risk, craft, and discipline together. The conclusion is that Opus 4.7 is not a categorical upgrade, but the direction is better.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Step 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis PerformanceQwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics