NVIDIA Cosmos 3: Four Robot and Autonomous Driving Simulation Models Released via Causal Distillation (Some Achieve Real-Time Performance)

NVIDIA has released four interactive world simulation checkpoints based on Cosmos3-Nano(16B) (Transfer-Auto, Sim-Auto, Sim-Transfer, Sim-Bimanual)
Bidirectional generative models were converted into causal generative models operating on short chunk units through a 4-stage Causal-rCM distillation process
Sim-Auto achieved real-time performance (400ms target) with 364ms/chunk (32.6 FPS) on a single GB200 under a single-camera condition, but did not meet the target under a 7-camera + LiDAR condition. Sim-Bimanual exceeded 30 FPS video speed with 48.1 FPS on a single GB300
The OpenMDW-1.1 license permits research and commercial use, and weights were released on Hugging Face on October 20, 2026

Technical Mechanism

Exposure bias was resolved and error accumulation prevented through 4-stage post-training (bidirectional adaptation, teacher forcing, consistency distillation, self-forcing)
Attention sink based on StreamingLLM was applied to limit KV cache size, and temporal causal attention was used to reference only past chunks
Training resources: 512 GB200s were used for Transfer-Auto/Sim-Auto training, and 768 were used for the consistency distillation stage

Transfer-Auto outperformed the previous generation in quality metrics such as FID and FVD, and Sim-Auto showed no quality degradation during 1-hour continuous generation
Sim-Transfer improved the success rate from 68.8% to 84.4% in the GR00T N1.7 policy, and Sim-Bimanual outperformed competing models on PAI-Bench-G I2V
Real-time performance measurements were conducted with pre-prepared control inputs, and verification of actual closed-loop latency and contact dynamics is required separately