Kakao's Test Generation Agent Case Study: LLM-Based Automated Unit Test Generation and Verification Process

LLM-based automated generation technology is gaining attention to reduce the burden of writing unit tests and improve quality. Through a case study of Kakao's test generation agent, we examine the process where the LLM analyzes method signatures and control flow to derive test scenarios, and configures Mock objects and assertions according to JUnit templates.

Automated Generation and Verification Process

The generated test code undergoes strict verification rather than simple output.

• Compile Verification: Syntax errors are blocked in advance via the Java Compiler.
• Execution Verification: Tests are actually executed with the JUnit Runner, forming a feedback loop where the AI analyzes logs and modifies the code upon failure.
• Coverage Measurement: JaCoCo is integrated to quantify coverage increases and display them as badges in PRs to ensure visibility.

Dependency Analysis and Similarity-Based Optimization

For accurate test generation, the AI analyzes the dependencies of the target class to configure appropriate Mock objects. Additionally, by applying similarity algorithms such as Levenshtein distance to detect existing test patterns and similar code within the project, it maintains consistent code style and prevents duplicate generation.

Development Workflow Integration Case

The agentic workflow presents scenarios integrated with development tools such as GitHub, Jira, and KakaoWork. It supports developers' quality management by automatically committing tests or suggesting them as review comments when changed classes are detected during PR creation, and providing notifications for high-risk changes (e.g., risk level 80% or higher).