Voice-Controlled React Component Built on OpenAI Realtime API

OpenAI has released a voice control component for React/browser that runs on top of the Realtime API.

It lets users direct form input or app actions via voice, and provides a tool-constrained UI pattern where the assistant is restricted to only performing actions the app allows.

The core structure is as follows.

• A single controller handles session connection, audio processing, tool execution, and transcript assembly all at once.
• Developers don't need to work directly with WebRTC or the Realtime protocol.
• When integrating into an existing app, the app maintains a single source of truth for state, and the voice layer only calls a restricted set of app handlers.

This package is not a general-purpose orchestration framework but a tool specialized for browser UI flows; for non-React runtimes or agent orchestration needs, using Raw Realtime or openai-agents-js is recommended.

An integration guide is also provided, covering:

• Proxying the /session endpoint
• Creating a voice adapter wrapper
• Registering tools
• Hoisting the controller
• Mounting the widget

The default setting is server_vad, and for tool-only sessions, interruptResponse: false is set so that an in-progress tool call isn't cancelled by new voice input.

The demo app showcases a variety of scenarios including theme switching, multi-step forms, a shared-state chess flow, and wake-word experiments, and it is licensed under Apache-2.0.