Dissecting AI Agent Evals

Good evals reveal problems and behavioral changes in AI agents before they go into production. Because agents make tool calls, change state, and adapt across multiple turns, they require much more sophisticated evaluation than simple single-turn tests.

When designing evaluations, you need to be clear about terminology first.

• task: a single test with defined inputs and success criteria
• trial: one execution attempt of the same task
• grader: logic that scores part of a transcript or outcome
• transcript: the full record including reasoning, tool calls, and intermediate results
• outcome: the final state actually left in the environment
• evaluation harness: the infrastructure that runs an eval end-to-end and records/aggregates results
• agent harness: the system that orchestrates a model to act as an agent
• evaluation suite: a collection of tasks with similar purposes

Agent evaluations often combine code-based, model-based, and human graders. Code-based graders are fast and reproducible but can be rigid, while model-based graders are flexible but non-deterministic and need calibration with human grading. Human graders are the most reliable but slow and expensive, so they're typically used for sample checks or aligning standards.

Evaluations broadly fall into two types. capability evals look at "what can it do well," and low pass rates are acceptable initially. regression evals, on the other hand, look at "does what used to work still work," so they need near-100% stability.

In practice, different agent forms like coding agents, research agents, computer use agents, and conversational agents can be evaluated under similar principles. In particular, coding agents have clear tests, making deterministic graders a good fit, and approaches like SWE-bench Verified or Terminal-Bench, which judge based on actual code execution and test pass/fail, are effective. However, pass/fail alone isn't enough, so code quality or tool usage patterns are additionally scored using LLM rubrics or transcript-based rules.

For example, a task that fixes an authentication bypass vulnerability can be evaluated by combining unit tests, static analysis, state checks, tool call verification, and an LLM rubric. Using outcome verification together with transcript verification like this lets you see both the agent's actual behavior and its quality at the same time.

This approach also matters for teams running real products like Claude Code, Descript, and Bolt. With evals in place, teams don't have to rely solely on user complaints — they can quantitatively manage model upgrade speed, latency, token usage, cost per task, and error rate. Ultimately, evals aren't just tests; they become the baseline that continuously drives agent development and improvement forward.