Amazon unveils Dialogue Boost, AI that makes TV and movie dialogue clearer

Amazon unveiled Dialogue Boost, which will be built into Echo smart speakers and Fire TV devices. This feature makes dialogue in movies, dramas, and podcasts clearer while adaptively suppressing background music and sound effects. It is especially useful for users with hearing loss, which Amazon noted affects about 20% of the world's population.

It was initially offered on Prime Video in 2022, but this version runs directly on the device using deep-neural-network compression. This allows it to be applied to any media, including Netflix, YouTube, and Disney+.

Dialogue enhancement is achieved through multiple stages of sound source separation.

• The input audio is first converted into a time-frequency representation that analyzes the energy of each frequency band over time.
• Then, a neural network trained on thousands of hours of speech conditions distinguishes speech from other sounds in real time.
• Finally, intelligent mixing is applied, combining dialogue-centric channel identification, source separation, emphasis of frequency bands critical to speech intelligibility, and remixing with the original audio.

The key elements are sub-band processing and pseudo-labeling. Instead of processing the entire frequency range at once as before, the audio spectrum is divided into sub-bands to enable parallel inference, and a lightweight bridging module improves consistency across bands. This architecture competes with previous state-of-the-art approaches while using less than 1% of the compute and only about 2% of the model parameters.

For training, rather than relying solely on synthetic data, the team used pseudo-labeling, where labels generated by the model itself on real media content are fed back into training. First, a large model is trained on synthetic data, then speech signals are extracted from real data, which is mixed back with synthetic data for retraining. Finally, knowledge distillation compresses the model into a small one capable of real-time execution, allowing it to run within device constraints.

The effect is clear. In discriminative listening tests, more than 86% of participants said Dialogue Boost was clearer than unprocessed audio, and a survey of users with hearing loss showed 100% feature approval. Amazon explained that it is especially useful in situations where dialogue tends to get buried, such as whispered conversations, various accents, action scenes, and nighttime viewing.