Apple Research Presents GHARP for Real-Time 3D Head Animation

• GHARP is a method for animating 3D human heads in real time from a few input images and a driving expression signal.
• The approach decouples identity reconstruction from animation, allowing a lightweight network optimized for mobile devices to run at runtime.
• It achieves state-of-the-art quality on the Ava-256 benchmark while running up to 13x faster on an A100 GPU with 8x fewer Gaussians.
• A body alignment network is introduced to resolve ambiguity in body pose and clothing, reducing blur and temporal flicker.

Methodology

• Identity stage builds subject geometry and appearance offline using a large-scale reconstruction prior.
• Animation stage predicts expression-dependent residuals in a semantically structured latent space.
• Fusion of multiple input views creates a compact, fixed-size canonical Gaussian representation.

Limitations and Solutions

• Expression codes omit body pose and clothing, causing underspecified regions in input.
• Standard expression-driven methods produce blur and flicker due to ill-posed mapping.
• Body alignment network aligns target body with reference images to remove training signal ambiguity.