The contract changed the performance

This is the result of running 52 controlled benchmarks over 3 weeks on a production Next.js/TypeScript/Supabase codebase. Sonnet 4.6 was used as the worker and Opus 4.7 as the grader, and all data and tools were made public.

The key lever was CONTRACT.md, written before coding. When a structured brief containing exact interfaces, column names, import paths, SQL rules, and non-goals was provided first, cost dropped by 54% and quality rose from 5/10 to 9/10. This result came from the same model and the same codebase.

In contrast, Agent Teams were inefficient.

• They were 73–124% more expensive than sequential execution, with no quality gain.
• Each agent independently loaded the full codebase context, greatly increasing cache costs.
• The explanation given is that with 3 parallel agents, it's effectively like paying for an 80K-token context 3 times.

Retry loops also backfired.

• Quality dropped from 9/10 to 6/10.
• During retries, the model repeatedly regenerated entire files instead of partially editing existing ones, breaking parts that were already correct.

The review stage also showed no benefit relative to its added cost.

• A single-pass Opus review did not further improve quality when the brief was already good, while costs rose by 56%.
• Conversely, Haiku, when implementing a contract written by Sonnet, achieved quality similar to Sonnet's at 64% lower cost, but when Haiku wrote the contract itself, quality collapsed to 4.9/10.

The codebase exploration structure also mattered.

• A 3-tier index of L0 summary → L1 signatures → L2 raw source outperformed a flat dump.
• Sequential workers achieved a 98% cache read rate on repeated context, while parallel workers had to refill the cache each time.

As a representative example, a $5.45 session was reduced to $0.83. However, it was noted that some N=1 results are for directional reference only, and a larger N=5 re-validation is also planned.