AI Briefing
KO

Over-editing: The phenomenon where models modify code more than necessary

·2026.04.23 02:51

Key point

It measured the problem of code-editing LLMs excessively rewriting instead of making minimal edits.

1 / 2

Details

This addresses the over-editing problem, where AI coding tools rewrite entire functions even when fixing just one bug.

  • In brownfield code that requires minimal edits, when a model makes more structural changes than necessary, review costs increase significantly.
  • The author programmatically contaminated 400 problems from BigCodeBench, designing the experiment so that the correct fix effectively becomes the minimal patch that reverts the original change.
  • The evaluation used token-level Levenshtein distance, Pass@1, and Added Cognitive Complexity.
  • As a result, even the latest models showed a tendency to over-edit. In particular, GPT-5.4 produced large diffs with a large increase in complexity, while Claude Opus 4.6 showed a high correctness rate with smaller diffs.
  • Adding an instruction to the prompt telling the model to "preserve the original code as much as possible" made all models edit less, and generally improved correctness as well.
  • By default, many reasoning models over-edited more than non-reasoning models, but when given the preservation constraint, they actually made more precise edits.

The key conclusion is that correctness alone is not enough — code-editing models need to be evaluated not just on whether they get the "right answer," but also on how little they change.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.