8 Indian languages via LoRA

After examining Chatterbox-Multilingual's T3 Transformer backbone, the author experimented with whether new languages could be attached using only LoRA, and tuned Telugu, Kannada, Bengali, Tamil, Malayalam, Marathi, Gujarati, and Hindi.

The trained parameters were 7.8M / 544M, just 1.4% of the total. The tokenizer was expanded from 2454 → 2871 tokens, and Indic character embeddings were initialized from their corresponding Devanagari characters using a Brahmic warm-start approach.

The core idea is that TTS can effectively be treated similarly to a transformer-based LLM.

• Adding a new language = partial fine-tuning + embedding expansion
• Stabilized initialization by sharing embeddings among characters with the same phonetic value
• Model and audio samples are released on HuggingFace

The base model is ResembleAI/chatterbox, under the MIT license. The author explains that Chatterbox seemed small yet expressive, but since some Dravidian and Indo-Aryan languages were missing, they decided to fill in the gap themselves.