World Model Based on Coding Agent Released

Researchers from the AGI Lab at Westlake University and Nanyang Technological University have released the Code World Model (CWM). This model proposes a new approach that separates world rules from screen generation to address the issue where existing video world models, which learn dynamics from observed frames, suffer from inconsistency during long-duration generation.

Core Working Principle

• Coding Agent (World Brain): Infers events and outcomes to write executable code, which continuously maintains the world state and creates changes according to the rules.
• Video Model: Renders actual screens conditioned on proxy videos (depth and semantic identifiers) generated by the code.
• Scalability: As the coding agent's reasoning and coding capabilities improve, the world expands more richly and consistently without retraining the model.

Technical Specifications and Limitations

• Proxy Representation: Each frame consists of a pair: a Depth file (float32, 192x336) and a Semantic-ID file (8bit PNG, 0–11 classes).
• Output: Generates silent H.264 MP4 files at 1344x768 resolution and 24 FPS.
• Hardware: Requires one NVIDIA H800 80GB GPU.
• Public Scope: Inference code, trained LoRA, and 40 examples are released, but tools for generating condition files from RGB videos are not included.
• License: The code follows the Apache License 2.0, but separate conditions may apply to third-party components and model outputs.