Voice Conversion

Voice Conversion is a technology that transforms one person's voice into another person's voice. Through Voice Cloning, it encodes the target speaker's voice, generating a message with the target speaker's identity while preserving the original intonation.

This technology has the potential to bring innovation to various industries.

• Film and Games: It can generate audio tracks without requiring actors to visit the set, or efficiently modify recorded dialogue.
• Healthcare and Personalization: It can help patients who have lost their voice due to illness communicate, or personalize virtual assistants with familiar voices.
• Advertising and Content: It enables the use of realistic synthetic voices while avoiding copyright issues, or optimizes audiobook and podcast production.

Eleven Labs plans to launch an identity-preserving automatic dubbing tool using this technology in early next year. The goal is to make it so that even when a user speaks in a language that is not their native tongue, it naturally sounds as if they are speaking it themselves, while preserving the original speaker's voice, emotion, intent, and style.

The technical principle is similar to face-swapping apps. The algorithm breaks down speech into its most basic units, phonemes, for training. Through this, it analyzes utterances in the source language and maps them to the appropriate intonation of the target language, preserving the speaker's identity and emotion.