The Journey Toward Real-Time Dubbing

The biggest challenge in real-time dubbing technology is resolving contextual uncertainty. In the case of garden-path sentences, where the meaning of a sentence changes as the speaker continues talking, the initial translation can turn out to be wrong, requiring a backtracking process to correct it.

Thus, real-time dubbing is a task of balancing latency and accuracy. Here, "real-time" refers to a service that continuously streams audio and returns translated content immediately, and some latency may be intentionally allowed to improve accuracy.

The most promising commercial application is live content aimed at global audiences, particularly sports broadcasts. Sports broadcasts already involve a certain amount of delay during the transmission process, so viewers are accustomed to a bit of additional lag.

Indeed, many producers have stated that they are willing to accept an additional delay of up to 20 seconds in order to provide high-quality dubbing in the local language.