Entering the Omniverse: How Open World Models Expand the Frontier of Physical AI

Physical AI must do more than recognize the appearance of objects; it must understand and predict how physical environments operate and the consequences of actions. World models learn environmental changes and next states to generate physically grounded world and action data, simulate future states, and provide the foundation for robotics, autonomous vehicles, and vision AI systems.

Data required for training physical AI is difficult to collect at scale, and rare events and long-tail scenarios are hard to reproduce safely and repeatedly. World models address these challenges in the following ways:

• Learn physical relationships from large-scale multimodal scenarios to generate more useful data.
• Create diverse environments by varying weather, lighting, objects, and movement trajectories.
• Provide a foundation that can be scaled to fit specific robots, vehicles, sensor configurations, tasks, and operational environments.

NVIDIA joined the open letter “Open Weights and American AI Leadership,” which emphasizes the importance of an open ecosystem, alongside more than 200 companies and institutions. NVIDIA Cosmos 3 is a suite of open world models that provides these capabilities and is being utilized in robotics, autonomous driving, and vision AI fields.

General models have not learned the specific robots, sensors, or operational environments used by each team, so real-world deployment requires model weights, licenses that allow model modification, and post-training tools. The Cosmos world foundation models are provided under the Linux Foundation’s OpenMDW 1.1 license, enabling teams to further train models using their own data and hardware.

Model specialization alone is not enough. Development teams also need an environment where they can generate data, run simulations, and validate actions. The NVIDIA Omniverse libraries support building environments ready for simulation, and OpenUSD provides an open framework for organizing, reusing, and exchanging complex 3D data in digital twin, simulation, and synthetic data generation tasks.