Continuous Improvement of the Agent Execution System

Cursor keeps tuning its agent execution system to match each model, treating dynamic context and experiment-driven improvement as its core principles. Early on, it raised quality with strong guardrails such as exposing lint/type errors, correcting file reads, and limiting tool calls, but now the direction has shifted toward letting the agent fetch more of the information it needs on its own.

Quality evaluation happens along two axes.

• Offline benchmarks like CursorBench are used to compare quality over time.
• On real user traffic, A/B tests are used to look at latency, token efficiency, number of tool calls, and cache hit rate.
• Outcome quality is measured via Keep Rate (the retention rate of generated code) and LLM judgments that read users' follow-up reactions.
• It was also concluded that more expensive context-summarization models bring only marginal quality gains, making them a poor cost-to-benefit tradeoff.

Tool errors can break an entire session, so they are managed separately. They are classified into categories such as InvalidArguments, UnexpectedEnvironment, ProviderError, UserAborted, and Timeout, and an alert fires immediately when unknown errors increase. Expected errors are monitored with anomaly detection against per-tool and per-model baselines, while weekly Automation scans the logs to ticket new issues and Cloud Agents run numerous fixes in parallel. This maintenance work has cut unexpected tool call errors by roughly 10x.

Per-model customization runs deep as well. Reflecting that OpenAI models are more comfortable with patch-based edits while Anthropic models are more comfortable with string replacement, Cursor provides different tool formats and prompts for each. When a new model arrives, Cursor starts from a similar existing execution system and tunes it through offline evals and real usage, and mitigates phenomena like context anxiety — where a model refuses tasks as context grows longer — through prompting.

Switching models mid-conversation is trickier. Cursor automatically swaps in the new execution system upon switching and adds instructions to prevent the model from mistakenly calling the previous model's tools. However, since caches are per-model and per-provider, the first turn's cost can spike, so for deep work Cursor recommends sticking with one model or starting in a fresh context, as with a subagent.

Ultimately, AI software development is heading toward multi-agent setups, where the execution system — deciding which agent to call when and how to weave the results together — matters more than any individual model.