How OpenAI Built a Real-Time System for Responsive Voice AI in Six Months

OpenAI redesigned the real-time system behind GPT-Live in six months so that voice AI can converse as naturally as a human. In existing voice AI, a small turn detector would first judge when the user finished speaking, and then the LLM would respond—if it acted too fast it would cut the user off, and if it acted too slow the response would feel sluggish.

GPT-Live adopts a full-duplex structure in which the voice model listens and speaks at the same time, removing the separate turn detector from the audio path. Audio is continuously fed into the model and the generated speech is streamed to the user, while large-scale reasoning or tool use is handled on a separate asynchronous path.

The core design separates the real-time media loop from the application logic. The path that keeps the voice flow going is separated from tasks like calling frontier models, storing conversations, and executing tools, so that expanding application functionality doesn't degrade responsiveness.

The system jointly optimizes the following elements:

• Stateful inference to support continuous conversation
• Dynamic context management to adjust the information needed for real-time voice processing
• Asynchronous delegation to call frontier models like GPT-5.5 and tools mid-conversation
• Protocol and media transport optimization to deliver audio frames at consistent timing

This architecture underpins features like ChatGPT Voice's computer control and agent coordination, forming the foundation for delivering both instant responsiveness in voice conversation and complex reasoning capability together.