NVIDIA Updates Cosmos Framework and NIM for Physical AI with 2-3x Inference Speedups

• NVIDIA released updates to the Cosmos platform, introducing the open-source Cosmos Framework for world model training and Cosmos NIM for optimized inference microservices.
• Cosmos NIM uses speculative decoding, quantization, and KV-cache-aware routing, with speculative decoding delivering measured mean speedups of roughly 2-3x on H200 GPUs.
• The framework supports Cosmos 3, NVIDIA Alpamayo, GR00T, Qwen, and pi0.5 models, with reinforcement learning paths including DPO, GRPO, PPO, OPD, and DiffusionNFT.
• Simulation recipes cover autonomous vehicles (30 fps, up to 400 frames), robotics (YAM and AgiBot embodiments), and surgical workflows (Open-H-Embodiment).

Performance and Deployment

• Streaming VLM inference provides sub-second alerts with 10x lower latency for live video understanding.
• Deployment requires Linux, NVIDIA driver, Docker, NVIDIA Container Toolkit, and an NGC API Key.
• Checkpoints are tested on NVIDIA Hopper and Blackwell GPUs with quarterly updates for features, optimization, and security.

Resource Requirements

• Reasoning VRAM ranges from 14.5 GiB (Cosmos3-Edge FP8) to 135 GiB (Cosmos3-Super BF16).
• Generation VRAM ranges from 14 GiB (Cosmos3-Edge-DROID BF16) to 150 GiB (Cosmos3-Super BF16), reducible to 38 GiB with FP8 and offloading.