AI Briefing
KOSign in

NVIDIA Research Introduces JCA for Joint Video and 3D Motion Generation

·2026.10.23 09:00

Key point

The model uses a separate motion branch connected via cross-attention to handle arbitrary skeletons and register motion in 3D scenes.

1 / 4

Details

  • JCA jointly generates video and synchronized 3D motion for rigs of any topology from a text prompt, unlike prior models limited to human skeletons or 2D pose triangulation.
  • The architecture separates the motion model from the video generator, connecting them with bidirectional cross-attention layers to allow the motion model to learn from motion-only data.
  • A joint-localization head predicts joint positions in video frames to recover the camera and register the 3D motion within the scene.
  • Training utilizes nearly 200K pairs curated from a generative video editor, human video, and robot video datasets including OiV, OpenVid, HAA500, HumanML3D, BONES-SEED, DROID, RoboCerebra, TAVIS, and ALOHA.
  • The model supports scene-aware animation, keyframe-controlled motion, video-to-motion recovery, and motion-to-video generation.
nvidiavideo-generationcross-attention3d-motionjcarig-animation
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

NVIDIA BlogKazakhstan Expands AI Compute with 400 H200 and 480 B300 GPUsHacker NewsNvidia OpenShell: Secure Runtime for Autonomous AI Agent FleetsHuggingFace BlogNVIDIA Releases SimReady Central: Open Library of 2,000+ Simulation-Ready 3D Assets on Hugging FaceHuggingFace BlogNVIDIA Releases Nemotron 3.5 Embed, Rerank, and Parse Models for Agent RetrievalNVIDIA BlogGermany Signs Joint Declaration with NVIDIA to Lead in Industrial AIGeekNewsArtCraft releases open-source IDE for AI image and video creation with 62-model catalog