Towards More Controllable AI Video Editing: Exploring Netflix's Early Research

Existing generative video editing models had a problem where, when modifying a specific element, the entire video would be regenerated, altering details that should have been preserved, such as a person's identity or the background. Additionally, when removing objects, physical continuity was often ignored, frequently resulting in unnatural motion.

To address these issues, Netflix has introduced two innovative research models.

• Vera: A Layered Video Diffusion Model that generates only the parts requiring modification as a separate layer while keeping the rest of the video as the original, thereby preserving the identity of the content.
• VOID: An Inpainting model for video object and interaction removal that goes beyond simply erasing objects, physically and plausibly reconstructing the surrounding environment as if the object had never been there.

Netflix aims to help artists precisely control their creative intent, and by releasing a paper detailing these research results and algorithms, seeks to contribute to advancement in the academic community.