Grok Voice Agent API Launches

xAI has unveiled the Grok Voice Agent API. It opens up the Grok Voice stack, previously used in the mobile app and Tesla vehicles, to external developers, enabling them to quickly build voice agents.

Performance was also emphasized. xAI trained its own VAD, tokenizer, and audio model from scratch, internalizing the entire voice stack, and stated it achieved #1 on Big Bench Audio. The average time-to-first-audio is under 1 second, which it described as nearly 5 times faster than competing products.

Pricing is a single plan based on connection time at $0.05 per minute. However, the text notes that when compared to OpenAI's billing method, token-based costs could be higher in production environments.

• Multilingual support: Speaks dozens of languages naturally, automatically responds in the user's language, and can switch languages mid-conversation.
• Evaluation results: In blind human evaluations against the OpenAI Realtime API, it claims to be preferred in pronunciation, intonation, and prosody categories.
• Tesla integration: Connects with tools for checking vehicle status, finding directions, and controlling navigation, and when planning trips, it performs X search, route calculation, and adding waypoints together.

Developers can use it in a way compatible with the OpenAI Realtime API specification, and the official xAI LiveKit Plugin is also available. Additionally, xAI provides a voice playground in the browser for testing various voices, and has previewed future plans for standalone text-to-speech and speech-to-text endpoints, as well as an audio model with better pronunciation and latency.