Nvidia Switchyard Router Cuts Costs to 1/3 by Reallocating AI Models During Tasks

A core challenge for enterprises running AI agents is the trade-off between cost and efficiency. Assigning all tasks to high-performance frontier models causes costs to skyrocket, while building custom routing logic increases maintenance burdens.

To address this, Nvidia released Nemotron 3.5 Lightning, a high-performance MoE (Mixture-of-Experts) model, and NeMo Switchyard, an open-source routing library.

Key performance and cost metrics are as follows:

• Nemotron 3.5 Lightning: Up to 4x faster output compared to models in the same class, and approximately 30% faster task execution compared to Qwen3.6-35B
• With Switchyard integration: Reduces costs to approximately 1/3 of the level of using Opus 4.8 alone, while maintaining frontier-level performance

The core of Switchyard is switching models in real-time based on the task's State. It selects the optimal model whenever the state changes, such as when an agent receives tool results or encounters errors. It also supports strategies that predict a model's Verbosity (token generation volume) to steer tasks toward cheaper models.

This solution integrates with agent frameworks like LangChain and Cognition, as well as existing LLM gateways like OpenRouter and LiteLLM, expanding the ecosystem.