Stable Video Materials 3D generated from a single image

SViM3D (Stable Video Materials 3D) is a framework that reconstructs 3D objects from a single image by predicting multi-view consistent PBR (materials) and surface normals.

Existing video diffusion-based 3D reconstruction is fast, but its material representation is simple or requires a separate estimation stage, which limits relighting and controlled appearance editing. SViM3D extends a latent video diffusion model to jointly output spatially varying PBR parameters and normals for each generated view, while precisely controlling viewpoints via explicit camera control.

Thanks to this design, the model goes beyond a simple reconstructor and is also used as a neural prior for generating relightable 3D assets. It also adds several mechanisms to improve quality in this incomplete and challenging setting, achieving state-of-the-art performance in both relighting and novel view synthesis across multiple object-centric datasets.

Key features are as follows.

• Generates consistent multi-view materials from a single image input
• Jointly predicts PBR parameters and surface normals
• Supports relighting through camera-controlled view generation
• Enables 3D asset generation usable in AR/VR, film, and games