Action Binding for Multiple Subjects in Video

Existing video generation models are vulnerable to the Action Binding problem, where they fail to accurately assign different actions to multiple subjects. For example, when given a command like "the red triangle moves to the right, the blue square moves up," the actions frequently get swapped or ignored.

ActionParty addresses this by introducing Subject State Tokens, which continuously capture the state of each subject, and a Spatial Biasing Mechanism. This separates global video frame rendering from individual subject action updates, enabling precise control.

This model is a generative game engine that operates based on a Diffusion Transformer (DiT). It denoises video frames while simultaneously predicting subject coordinates, and maintains clear correspondence between actions and subjects through Attention Masking.

On the Melting Pot benchmark, ActionParty demonstrated the ability to simultaneously control up to 7 players across 46 diverse environments. This shows superior performance compared to existing models in terms of action execution accuracy and subject identity preservation.