Conversation Doesn't Wait Its Turn: 'Sommelier,' a Data Pipeline for Real-Time Voice Conversation AI

Real-time voice conversation AI must respond instantly without waiting for the user's utterance to finish, so conventional sequential processing methods have limitations. To solve this, NAVER designed a new data pipeline called Sommelier.

Sommelier achieves real-time performance through the following core mechanisms.

• Streaming-based processing: Splits voice data into chunks for processing to reduce latency.
• Asynchronous pipeline: Manages each stage (STT, LLM, TTS) as an independent stream to maximize parallel processing.
• Dynamic orchestration: Flexibly controls data flow according to conversation context and network conditions.

This system organically connects the speech recognition (STT), generation (LLM), and speech synthesis (TTS) processes, aiming to achieve a natural response speed that feels like conversing with a human.