Introducing Cosmos 3 Edge

NVIDIA Cosmos 3 Edge is a 4 billion (4B) parameter open world model designed to enable robotics and vision AI agents to perform real-time inference and action generation in edge environments. It has been released via Hugging Face and supports NVIDIA edge computers broadly, including RTX PRO GPU and Jetson Thor.

The model consists of two transformers: an autoregressive tower and a diffusion tower. The autoregressive tower processes vision and text tokens to handle understanding and reasoning, while the diffusion tower processes vision, audio, and action tokens to handle prediction and generation. The two towers share multimodal attention layers to align information across language, video, audio, and action.

On Jetson Thor, it generates 32 actions per inference and achieves 15 Hz real-time control. It ranked #1 among comparable 4B models on the VANTAGE-Bench vision analysis benchmark.

Actions are encoded as a common geometric vector including Translation, Rotation, and Manipulation state, unifying diverse physical systems such as vehicles, cameras, and robot arms into a single representation.