Wan-Dancer: Generating Dance Videos Over 1 Minute Long

A new hierarchical framework called Wan-Dancer has been announced to solve the temporal drift and motion repetition problems that existing Diffusion Models experienced when generating videos longer than 20 seconds.

This model separates the process into Global keyframe planning and Local temporal refinement stages, maintaining long-term consistency while preserving the context of the entire song.

The key technical features are as follows:

• Time-mapped RoPE embeddings: Achieves precise alignment between music and motion through dynamic frame rate adaptation
• Optical-flow-based loss: Applies an optical flow-based loss function to enhance motion continuity
• Motion-speed control: A speed control feature that maintains high-resolution detail even during fast movements

Experimental results show that it generates stable videos over 1 minute long at 720p/30fps resolution, overcoming the temporal limitations of existing models. The model weights and inference code are currently available via GitHub and HuggingFace.