AI BriefingPrism: Simultaneous 2K Video and Audio Generation, 2.5x Faster Training
FrancisRing/Prism
About the project
Prism is a Joint Video-Audio Generation model that simultaneously generates video and audio at 2K resolution. It synthesizes sound alongside visuals, maintaining visual detail and dynamic motion even in high-resolution environments that were difficult for previous methods to handle.
The core technology is Dynamic Sparse Attention. It divides token sequences into spatiotemporal macro zones and analyzes the distribution of video features and the strength of audio-visual interactions to assign optimized block shapes to each region. This reduces unnecessary computations while maximizing the transmission of meaningful information.
Training speed is increased by 2.5x compared to full Attention methods, with generation quality surpassing that baseline. It supports various resolutions from 720p to 2K, and results can be obtained by inputting only an image and a prompt via the Diffusers library.
Developed by researchers from Fudan University and Tencent Hunyuan, this model is released under the MIT license. It is a suitable choice for developers building high-resolution multimodal generation tasks or large-scale training pipelines.
FrancisRing/Prism
The original page has no description.
image-to-video
This introduction was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report errors, attribution issues, or removal requests via Contact.