The End of Traditional Testing: Agentic Development Broke 50-Year-Old Testing, and JiTTesting Might Revive It

As agentic software development makes code get written, reviewed, and deployed faster, existing testing methods now have to keep pace with that speed. Approaches relying on static test suites and manual maintenance hit their limits in the face of rapid changes and frequent false positives.

JiTTests(Just-in-Time Tests) is an approach in which an LLM generates tests on the fly at the time of a pull request. Catching JiTTests, which this piece covers, focuses specifically on finding regression caused by code changes, and is described as reducing the need for humans to write, review, or continuously maintain test code.

The workflow is as follows.

• New code enters the codebase.
• The system infers the intent of the code change.
• It creates mutant(a version of the code with an intentionally introduced defect) versions to see what could go wrong.
• It generates and runs tests to catch that defect.
• rule-based and LLM-based assessors refine the signal to focus on true positives.
• Engineers receive a clear report only when an actual bug is found.

The core value lies in turning tests from "a tool for broadly checking code quality" into "a tool for finding actual defects in specific changes." The claim is that this reduces maintenance costs and the burden of false positives, while building a testing infrastructure that matches the speed of agentic development.