4Director Controls Video World Models with Rigid 3D Geometry

Precise control over camera and object motion remains a challenge in professional video production, as existing methods rely on ambiguous image-plane cues or incomplete 3D tracks. 4Director addresses this by conditioning a video world model on an explicit 4D scene representation, where each object is reconstructed once as a canonical mesh and moved by a prescribed rigid transformation per frame.

This approach prevents unobserved geometry from being regenerated independently in every frame, ensuring consistency across viewpoint changes. The system renders the controlled scene as a depth video and employs a Motion Adapter to transform this geometric scaffold into video, synthesizing view-consistent appearance, illumination, and non-rigid dynamics.

Training and Evaluation

To support this method, the authors constructed RealCOD-Rigid, a new dataset containing 20,774 clips annotated with rigid 3D scenes via an automatic pipeline. They also introduced Identity-Gated IoU (IG-IoU), a metric that jointly evaluates adherence to prescribed object motion and the preservation of object identity.

Experiments demonstrate that 4Director consistently outperforms prior methods in both visual quality and the accuracy of camera and object control.