SlopCodeBench Study: AI-Generated Code Has Twice the Duplication and Complexity of Human Code

As AI coding tools become widespread, code generation accuracy has improved, but issues of 'sloppiness'—such as unnecessary abstraction, duplication, and poor decision-making—have emerged. Sebastian (Earendil), drawing on a background in physics, proposed the SlopCodeBench framework to quantitatively measure this degradation in code quality.

New Metrics: Verbosity and Erosion

To overcome the limitations of existing 'vibes based' evaluation methods, two objective metrics were introduced.

• Verbosity: Measures the ratio of lines flagged by AST-Grep and duplicate (clone) lines. The formula is (AST-Grep flagged lines ∪ clone lines) / LOC.
• Erosion: Indicates how concentrated the codebase's mass is in complex and large functions. It is defined as mass(f) = CC(f) * √SLOC(f) (CC: Cyclomatic Complexity, SLOC: Source Lines of Code) and calculated as the sum of mass for functions with CC>10 divided by the total mass.

Evaluation Results: Confirmed Inferior Quality of AI Code

Comparing existing legacy repositories (Repos) with code generated by AI agents (Agents) using SlopCodeBench, AI code was found to be on average approximately 2x more verbose and eroded than human code.

• Verbosity: Repos 0.15 ± 0.06 vs Agents 0.33 ± 0.10
• Erosion: Repos 0.31 ± 0.17 vs Agents 0.68 ± 0.20

Additionally, tests simulating the iterative coding process of real human developers—by providing instructions over multiple rounds and clearing context—showed that even the latest SOTA models like GPT 5.6 sol xhigh recorded a Strict solve rate of 0% across all checkpoints. This suggests that AI-generated code can suffer from accumulated poor decisions, leading to a rapid decline in quality over time.