AI BriefingReal-Time Speech-to-Text (STT) Under 200ms
Key point
Covers architecture design and communication protocol comparisons for building ultra-low-latency real-time STT systems.
Details
Reducing the latency of real-time STT (Speech To Text) is not just a matter of model performance—it requires optimization across the entire architecture, including transmission, chunking, and endpointing.
ElevenLabs' Scribe v2 Realtime records a model latency of about 150ms, supporting over 90 languages and PCM/mu-law audio formats. It also supports segment finalization through VAD (Voice Activity Detection) and manual commit control.
For audio transmission methods, WebSocket and WebRTC stand out as the main options.
- WebSocket: Based on TCP, it offers high reliability and simple implementation, making it suitable for most server-to-server communication or broadband environments, but Head-of-line blocking can occur when packets are lost.
- WebRTC: Based on UDP, it is resilient to packet loss and has low latency, making it advantageous in unstable environments such as mobile networks, but server-side implementation complexity is higher.
For an efficient system, small PCM chunks of about 100ms should be used to minimize the time it takes for the first partial text to be delivered.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.