AI BriefingWhat Percentage of Agent Calls Actually Require a Frontier Model?
Key point
Switchyard handled 93% of agent calls with a 30B model, reducing costs by 74%.
Details
NVIDIA NeMo Switchyard is an open-source routing library that automatically selects the appropriate model for each step of an agent workflow. Instead of sending all calls to frontier models, it processes simple tasks with cheaper open models and routes only complex tasks to high-performance models.
When applied to the Deep Agents evaluation suite, only 7% of total calls were routed to frontier models. The remaining 93% were handled by NVIDIA Nemotron 3.5 Lightning, a 30B parameter model.
Compared to using Claude Opus 4.8 alone, the routing approach combining Nemotron and Opus reduced total costs by 74% while maintaining approximately 93% accuracy relative to Opus-only performance for the same calls.
Switchyard can run as a proxy connected to agents or as middleware within the agent process, supporting configurations that combine closed and open models. Routing strategies are selected based on the balance between cost, accuracy, and latency.
The decision to adopt routing should first be based on calculating the price difference between models and the actual offloading ratio. If the prices of the two models are similar, the offloading ratio required to break even exceeds 100%, making routing economically unviable; cost savings may only be possible when directly hosting cheaper models.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.