Code-as-World: Restoring Videos as Executable Physics Code, Surpassing Gemini 3.1 Flash with a 9B Model

The MirroS team released the 'Code-as-World' research to address the limitations of Vision-Language Models (VLMs) in quantitative physical reasoning by restoring videos into executable physics code rather than pixels. While existing pixel-based approaches failed to distinguish the causes of changes, resulting in inaccurate restoration of physical quantities, this research uses code specifying objects, relationships, and physical parameters as the world representation to generate accurate state trajectories.

Executable World Representation (EWR) and Agentic Loop

The proposed Executable World Representation (EWR) consists of three parts: physical configuration, dynamic evolution, and visual appearance. The authors defined the process of restoring EWR from observations as an agentic discovery loop that iterates through hypothesis proposal, instantiation, execution, rendering, and verification. This loop utilizes tools such as SAM 3 and VGGT-Omega to extract depth, masks, and 3D geometry from videos, minimizing errors by comparing simulator execution results with input videos.

Code-as-World-VL Model and Performance

The research team released Code-as-World-VL 4B and 9B models trained on physical quantities extracted from verified executable worlds. Stage 1 involved supervised fine-tuning (SFT) for image plane measurement capabilities, while Stage 2 employed GRPO reinforcement learning using world-space questions. Benchmark results reported that the 4B model outperformed 32B-class open-weight models, and the 9B model surpassed the performance of Gemini 3.1 Flash.

Release Scope and Limitations

The current repository includes inference code, QuantiPhy evaluation scripts, a ballistic soccer example, and 4B/9B checkpoints. However, the complete implementation of the agentic discovery loop, video generation models, the 27B inference variant, and the executable world learning dataset have not yet been released.