Magenta RealTime 2: Open Source and Local Real-Time Music Model

Magenta RealTime 2 (MRT2) is a state-of-the-art open model and efficient real-time inference engine that can turn a laptop into an AI instrument.

Unlike existing offline generation models, it interacts in real time via MIDI, text, and audio, providing instant responsiveness through low-latency, on-device inference.

Key features and components include:

• An open-weight model with 2.4B parameters.
• Achieves a 40ms frame size and ~200ms control latency, making it usable like an actual instrument.
• Supports efficient streaming audio generation using MLX on MacBook (Apple Silicon) environments.
• Release of an open source Python library and C++ inference engine based on JAX/MLX.

Users can run it as a standalone app or integrate it into a DAW (Digital Audio Workstation) to use for sound cloning, style blending, real-time accompaniment generation, and more.