Developers Use GPT-6 Astra and NVIDIA Omniverse to Build AI-Driven Simulations

Developers are combining frontier AI models with NVIDIA Omniverse libraries to accelerate the creation of complex simulations. By directing GPT-6 Astra with natural-language instructions, they can assemble assets, connect physics and rendering, and validate scene behavior without manually coding every integration.

Humanoid and Autonomous Driving Simulations

Frank DeLise, an Omniverse product manager, used Astra to build a humanoid simulator for a warehouse environment. The agent connected ovphysx for physics, ovstage for scene updates, ovrtx for rendering, and ovui for the interface, generating the necessary application code to control a robot in first- and third-person views.

In another project, Doyub Kim directed Astra to create Zero to Alpamayo, a reusable simulation environment based on San Francisco’s Market Street. The workflow integrated asset creation, traffic, and Alpamayo driving models. A separate Cosmos3-Nano experiment varied weather and lighting to compare driving model responses under different conditions.

Digital Twins and Robot Testing

Ashley Reid used Astra and Claude Fable 5 to create and refine digital twins by comparing simulated camera and LiDAR outputs with recorded data. Over three days, the agents modified OpenUSD scenes to address geometry and material discrepancies, guided by sensor metrics.

Tae Kim led the development of Robo Olympics, where Astra built controllers for Unitree G1 humanoids using the Newton Physics Engine and NVIDIA Warp. In one test, the simulated robot cleared a hurdle in 64 of 100 trials, providing feedback for timing and control improvements.

Design, Tooling, and Web Applications

Jens Jebens used Astra to model a car suspension in PTC Onshape and configure it in NVIDIA Isaac Sim. The agent designed a custom wrench for a robot to reach suspension bolts, successfully simulating the disassembly process.

Nic Johns prompted Astra to assemble NASA assets into an International Space Station model with telemetry, using Blender for preparation and Omniverse libraries for streaming to the browser. Chirag Majithia directed Astra to convert stereo camera captures into editable OpenUSD studios, using PyCuSFM, FoundationStereo, and nvblox for reconstruction and USD Content Agents for interaction testing.