Verifying Agent Behavior When There's No Single Right Answer

In agentic environments where there isn't a single fixed correct answer, existing tests easily break down. Computer Use-based systems like GitHub Copilot Coding Agent interact with UI, browsers, and IDEs, absorbing loading screens, timing differences, and path branches as normal variation, but GitHub Actions workflows can't keep up with this, creating false negatives that judge actual successes as failures. In this process, fragile infrastructure and the compliance trap also recur.

The limitations of existing approaches are clear.

• assertion-based testing breaks with even slight differences in execution path.
• record-and-replay is vulnerable to rendering and timing noise.
• visual regression testing only compares screens without understanding meaning.
• ML oracles have major problems with training data and explainability.

The key is an independent Trust Layer. It views execution as a graph rather than a script, building a ground truth model of normal behavior from just 2 to 10 sessions that actually succeeded. Like a VS Code search, whether or not a loading screen appears, reaching the Search Results state is ultimately considered a success.

The core procedure consists of three steps.

• Build a graph of successful traces using a PTA (Prefix Tree Acceptor).
• Merge states using a three-stage equivalence determination framework. First, filter out near-identical states with perceptual hash and SSIM, and only when it's ambiguous does a multimodal LLM determine the semantic difference.
• Use dominator analysis to extract the essential states that必ず appear in every successful path, organizing these into a dominator subtree and excluding optional variations like loading spinners.

This approach creates a verification standard that is explainable and lightweight without manual specifications or large-scale training, and can be put directly into actual CI pipelines. The standard for verification shifts from what was observed to happen, to what had to happen for success to occur.