How OpenAI Built GPT-Live

GPT-Live removes the turn detector from the audio path of existing voice AI and applies a full-duplex voice model that can listen and speak simultaneously. This eliminates the wait for a separate model to determine if the user has finished speaking, thereby improving conversational responsiveness.

When deeper reasoning or tool use is required, GPT-Live asynchronously invokes frontier models such as GPT-5.5. The core voice path remains uninterrupted, while tasks such as reasoning, tool invocation, and conversation storage are handled on separate paths.

To achieve this, OpenAI redesigned model inference, context management, and media transport over six months. They built a stateful inference system that streams audio from the client to the model, delivering every audio frame at a consistent point in time to reduce voice gaps and interruptions caused by transmission and processing latency.

The architecture separates media flow from application and business logic.

• Audio moves between the client and the voice model via a dedicated high-speed path.
• Delegation, tool use, and backend tasks are handled through asynchronous RPC boundaries.
• Slow tool calls or backend services cannot block the real-time voice flow.

This structure allows for independent customization of application behavior while maintaining responsiveness. OpenAI is also leveraging this foundation for computer control in ChatGPT Voice and agent collaboration features in the ChatGPT desktop app.