Noiz AI and HKUST Release Helix World, a Real-Time Audio-Visual World Model

Noiz AI and The Hong Kong University of Science and Technology (HKUST) have introduced Helix World, a real-time interactive audio-visual world model that generates explorable scenes and spatial stereo sound simultaneously. Unlike most silent world models, Helix World conditions audio generation on camera motion, ensuring the sound field rotates with the user's perspective. The model runs at 24 frames per second with a real-time factor of 0.77 on a single NVIDIA H800 GPU, allowing users to navigate environments using WASD and arrow keys.

Benchmark Performance

On the WBench navigation split, Helix World scores 79.9, the highest among models running faster than real time. It also introduces HelixBench, a new benchmark for interactive audio-visual models with 1,015 human-verified cases. On HelixBench, Helix World achieves a spatial-acoustic consistency score of 41.8, significantly outperforming the baseline EchoWM at 12.7. In user preference studies with audio enabled, participants preferred Helix World over EchoWM 67.9% to 20.8%.

Architecture and Training

Helix World builds on the open-source LTX-2.3 audio-visual model. It incorporates continuous 6-DoF camera trajectories via PRoPE and discrete keyboard actions via AdaLN-Zero. Training involves a bidirectional teacher model distilled into a few-step causal student to enable streaming without drift. The team curated a dataset of 3,000 hours of video with true stereo audio and metric camera poses from an initial pool of 4,100 hours.

Availability and Limitations

The model is currently available as a preview release with code on GitHub and weights on Hugging Face. The interactive demo is hosted at helixworld.org. Limitations include an output resolution of 768×512 and a reliance on data-center GPUs for real-time performance. While it excels in audio-visual alignment, it trails slightly behind the best silent models in pure visual quality scores.