AI BriefingMeasuring Agent Aging, AgingBench Released
Key point
AgingBench, a benchmark that measures the phenomenon of deployed AI agents' performance degrading over time, has been released.
Details
A new benchmark, AgingBench, has been released to study and measure the 'Aging' phenomenon in which AI agents' performance degrades over time after deployment.
According to the research, in the Claude Code CLI environment, when the backbone model was swapped from Sonnet 4.6 to Opus 4.7, PyTest pass rates decreased by about 15%. This shows a paradoxical situation in which performance drops despite the model's higher base capability.
This performance degradation is analyzed not as a lack of capability in the model itself, but as a longitudinal effect occurring during long-term operation. The main causes are as follows:
- Changes in memory state: Memory compression, interference, modification, and maintenance shocks that occur as sessions repeat.
- The importance of memory policy: Experimental results showed that memory policy had about 4.5 times more impact on an agent's half-life than model swapping.
In conclusion, this suggests that when building long-running agent systems, simply swapping in the latest model is not always a safe upgrade strategy, and memory management policy plays a key role.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.