AI BriefingSlopCodeBench Finds Coding Agents' Code Quality Worsens in Most Runs Across Evolving Checkpoints
·2026.10.19 12:05
Key point
SlopCodeBench, a UW–Madison benchmark, tests 15 agents on evolving coding tasks and reports that structural erosion increased in 77% of runs.
Details
- SlopCodeBench (arXiv 2603.24755, UW–Madison) tests 15 coding agents on problems that evolve over multiple checkpoints, with no earlier chat history, only the codebase.
- None of the 15 models fully solved any of the 36 problems; best strict checkpoint solve rate was 14.8% (GPT 5.5), and Opus 4.6 had the highest core test pass rate at 67.3%.
- Code quality worsened in most runs: structural erosion increased in 77% of runs and verbosity in 75.5%, with agent code 2.3x more verbose than code from 473 human-maintained Python repos.
- On one problem, Opus 4.7's main() grew from cyclomatic complexity 13 to 92 over 8 checkpoints (38 to 240 lines), getting worse at every step.
Prompting effects
- Anti-slop and plan-first prompts cut initial verbosity and erosion by up to about a third.
- They did not stop degradation over time and raised cost per checkpoint by about 12% on average.
- These prompts were only tested on GPT models.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Hybrid Reasoning Agents achieve 35-55% token reduction and up to 2.6x faster inference
StarSkirmish Bench v0.2: GPT-6 Astra Leads LLM StarCraft Bot Coding, Human Bots Still Dominate
More Agents, Same Mythical Month: AI Amplifies Organizational Coordination Gaps