Testing Distributed Systems Using AI Agents

This describes two core skills (SKILL.md) that enable AI coding agents (Claude Code, Cursor, Gemini, etc.) to design and execute claim-driven tests for distributed and stateful systems.

This methodology separates test design from test execution.

• Design skill: Starting from the product's claims, it generates hypotheses and writes scenarios to disprove each hypothesis. For scenarios where consistency matters, it also defines an abstract model, an operation history schema, and a checker.
• Execution skill: It executes the designed plan and produces a result report with a verdict in one of 9 states, classifying whether the issue lies in the SUT (System Under Test), the harness, the checker, or the environment.

Unlike conventional simple integration testing approaches, this method has the following characteristics.

• Claim-centric: Tests are named based on the product's claims rather than configuration, making the purpose of each test clear.
• Coverage adequacy: At the planning stage, it argues whether the selected scenarios are sufficient and explicitly states what remains unverified.
• Use of models and checkers: Going beyond simple chaos engineering, it uses concrete models and checkers such as linearizability and serializability to verify safety and durability.
• Clear responsibility classification: When a failure occurs, the cause is classified into SUT, harness, checker, or environment, optimizing the reproduction and fixing process.

The final deliverable is provided as a structured Markdown file, allowing reviewers to decide whether to ship based solely on the plan and result report, without having to re-run the tests themselves.