Service Web Development Team Preemptively Blocks 1,717 Defects Using ATDD and AI Agents

The Service Web Development Team conducted an experiment combining ATDD (Acceptance Test-Driven Development) and AI agents (Claude Code) during the development of multi-category selection and dynamic filters. By utilizing 197 test cases (TCs) delivered by QA before development completion as automated specifications during the development phase, 1,717 defects were preemptively blocked before QA started.

Dual-Loop Structure of TDD and ATDD

The experiment was conducted by clearly separating the developer's unit test loop (TDD) and QA's acceptance criteria loop (ATDD). QA first finalized acceptance criteria written in user language, and developers repeatedly ran the red-green-refactor cycle in their internal logic to pass these criteria. This process resolved ambiguous requirements into concrete examples before code was written, reducing interpretation errors.

Role of AI Agents and Boundaries of Human Judgment

Claude Code was deployed to convert and verify the 197 TCs into test code. The agent performed draft creation for moving TCs into test code, comparison of the entire code against API responses, and rebuttal verification (101 requests) to identify implementation errors. However, the final decision on whether discovered discrepancies were implementation bugs or TC errors was made directly by developers who understood the project context.

Experiment Results and Limitations

As a result of the experiment, approximately 95% of TCs passed on the first run when QA initially executed them, and no defects occurred after the service launch. However, the pre-agreed goal of "fewer than 10 defects during the QA period" was missed with 14 defects. The author evaluated that tracking the causes of these 14 defects was the biggest harvest of the experiment, as it enabled data-driven prioritization of quality investments.