How Descript Built Large-Scale Multilingual Video Dubbing Technology

Descript, an AI-native video editor, has long relied on OpenAI's Whisper and GPT models to support complex workflows such as transcription, editing, and audio cleaning. Recently, the company has been focusing on high-quality multilingual dubbing technology that goes beyond simple subtitle translation and preserves the natural flow of the video.

The most challenging problem in dubbing is Duration Adherence, since the number of syllables needed to convey the same meaning differs from language to language. For example, German requires more syllables than English, so when the existing approach translated first and then tried to fit the length afterward, the resulting speech would become unnaturally fast or slow.

To solve this, Descript redesigned its translation pipeline using OpenAI's reasoning models. Previously, translation was performed first and length adjustment was attempted afterward, but the new system optimizes both semantic fidelity and duration adherence simultaneously from the generation stage.

In particular, by leveraging the enhanced reasoning capabilities of the GPT-5 series models to precisely count syllables and track constraints, the system was designed to naturally adjust length during the translation process itself. As a result, within 30 days of rollout, dubbing exports increased by 15%, and duration adherence improved by 13-43 percentage points depending on the language.