The Problem of False Compliance in AI Agent Evaluation via String Matching and the Need for Direct Verification

Structural Flaws in String Matching Evaluation

In AI coding agent evaluation, the string contains method is fast and cheap, but it only checks for the presence of strings regardless of actual code execution or build success, which can lead to false assurance. This is because evaluation can pass if the technology name is found in comments, dead code, or unused dependencies. Conversely, the absence of a string does not mean the technology was not used. This confuses the observed fact (string presence) with the actual implementation of functionality.

Limitations of LLM Judges and the Principle of Direct Verification

LLM judges are useful for distinguishing semantic criteria but cannot verify deterministic attributes such as compilation success or dependency resolution. There are cases where code receiving a perfect score from an LLM judge actually fails to compile. Therefore, buildability should be verified with a compiler, JSON validity with schema validation, and app behavior with actual execution. LLM judges cannot replace these deterministic tools; appropriate tools must be used for semantic questions and deterministic questions respectively.

Designing Reliable Agentic Eval

Agent capability evaluation should be treated as integration tests, not unit tests. Build, test, run, and deploy should be set as separate gates, and the causes of failure at each stage must be clearly distinguished. Evaluators must verify whether evaluation metrics actually support their claims by asking, 'What can we definitely know if this evaluation passes?' and, if full execution is impossible, clearly report what the evaluation has and has not established.