Is the Cost of AI Agents Also Rising Exponentially?

METR's graph shows that the length of tasks AI agents can handle has been rapidly increasing over the past 7 years, but the key question is at what cost that performance comes. Model size has grown roughly 4,000x, output tokens roughly 100,000x, and the inference cost required to achieve top performance has likely risen along with it.

The author proposes hourly cost as the key metric: the cost of performing a task at each model's 50% time horizon, divided by the human work time. For example, if Claude 4.1 Opus performs a 2-hour task at a 50% success rate, you divide that cost by 2 to get the hourly rate.

The problem is that METR's total benchmark cost doesn't directly equate to economic cost. Because METR runs the scaffold for a long time until performance reaches a plateau, some models end up using far more compute than actually necessary.

So the author draws constant hourly cost lines on METR's GPT-5 cost-performance graph to find the sweet spot and saturation point. The sweet spot is the point with the cheapest hourly cost, and the saturation point is where the slope drops to 1/10 of the sweet spot's slope.

The analysis found that the lowest rate for a human software engineer was about $120/hour, while AI models varied widely, from about $40/hour for o3 down to $0.40/hour for Grok 4 and Sonnet 3.5. However, near the plateau, costs jump back up by 10-100x — for example, GPT-5 costs about $13/hour on a 45-minute task but reaches about $120/hour on a 2-hour task.

In conclusion, while the growth rate of time horizons alone looks impressive, the cost of obtaining that capability is also growing in tandem. This means METR's trend may not be simply a rise in capability, but rather a result inflated by increasingly large amounts of inference compute — and real-world AI agent adoption is likely to track far behind this frontier-performance trend.