Up to 30x Work Throughput per Watt: NVIDIA Vera Rubin NVL72 Sets New Record for AI Agent Efficiency

AI agent workloads consume 15x more tokens than simple chat requests. This is because context accumulates as agents repeatedly perform tasks such as financial database queries, news searches, and sub-agent calls. Handling such long contexts has become a key factor determining the efficiency of AI infrastructure.

NVIDIA announced that the Vera Rubin NVL72 system delivers up to 30x higher throughput per watt for agent workloads compared to the GB300 NVL72. This result was measured using the SemiAnalysis AgentX workload, which records actual coding sessions. In power-constrained AI factories, this means 30x more agent tasks can be performed with the same energy consumption.

The Vera Rubin NVL72 also offers up to 35x lower cost per token compared to the GB300 NVL72. This significantly improves profitability when running agents continuously at scale. NVIDIA explained that through DSX MaxLPS technology, power is managed at the GPU, rack, and workload levels, allowing up to 40% more GPUs to be provisioned within the same megawatt budget.

These performance improvements are due to Extreme Codesign across the entire platform. The key optimization techniques are as follows:

• Disaggregated serving: Separating context processing (prefill) and response generation (decode) to scale them independently
• Rate matching: Synchronizing token generation speeds of prefill and decode GPUs to maximize efficiency
• Large-scale expert parallelism: Distributing expert subnetworks of MoE models across GPU domains
• Distributed KV-caching: Expanding memory within GPU domains and offloading inactive contexts to host/storage
• KV-aware routing: Routing requests to GPUs that already hold the relevant cached context to prevent redundant computation