VibeVoice - Open-Source Frontier Voice AI Model

Microsoft has released VibeVoice as an open-source voice AI model family. It provides both ASR and TTS, combining an ultra-low 7.5Hz frame-rate continuous speech tokenizer with next-token diffusion to improve efficiency in processing long audio.

• VibeVoice-ASR (7B) processes up to 60 minutes of audio in a single pass.

• It works like Whisper, but with speaker diarization built into the model, and output is organized in a Who / When / What structure.

• It supports custom hotword and natively supports 50+ languages.

• It has been integrated into Hugging Face Transformers since March 2026, and vLLM inference and fine-tuning code have also been released.

• VibeVoice-TTS (1.5B) generates up to 90 minutes of conversational speech at once, supporting up to 4 speakers.

• It aims for natural speech that reflects emotional nuance and conversational dynamics.

• However, unintended use cases were identified, and the TTS code was removed from the repository on September 5, 2025.

• VibeVoice-Realtime (0.5B) is a model for real-time TTS up to 10 minutes long.

• It takes about 300ms to produce the first voice output and supports streaming text input.

• As of December 16, 2025, 9 languages and 11 English-style voices were experimentally added.

• Apple Silicon (MPS) support was also added to the Gradio ASR demo.

Since the base model is Qwen2.5 1.5B, it may inherit its biases and errors, and the risk of deepfake misuse remains. The license is MIT.