Stable Part Diffusion 4D: Generating Multi-View RGB and Kinematic Part Videos from a Single-View Input

Stable Part Diffusion 4D (SP4D) is a framework that generates RGB frames and kinematic part segmentation together from a single-view input. Rather than relying on appearance-centric semantic segmentation as before, it learns parts that align with an object's articulation structure and temporal consistency.

The core is a dual-branch diffusion model. One branch generates RGB video, and the other generates corresponding part segmentation maps, designed so that the two outputs stay aligned with each other.

To simplify the architecture, it introduces spatial color encoding that converts part masks into continuous RGB-like images. This allows the segmentation branch to share the latent VAE of the RGB branch, and the final part map is recovered with only post-processing.

Consistency between branches is reinforced by the BiDiFuse (Bidirectional Diffusion Fusion) module. A contrastive part consistency loss is added on top of this to promote spatial and temporal alignment.

• The generated 2D part maps can be lifted to 3D.
• This allows skeletal structure and harmonic skinning weights to be extracted with relatively little manual effort.
• The number of parts can be flexibly varied, making it easy to apply to diverse objects.

For training and evaluation, KinematicParts20K was built. It was curated and processed from Objaverse XL, gathering 20K+ (over 20,000) rigged objects, each paired with multi-view RGB and part video sequences.

Experiments show that the method generalizes broadly to real footage, newly generated objects, and even rare articulated poses, producing kinematic-aware output that can be used in downstream animation and motion-related tasks.