Lexical Intervention for Multilingual Knowledge Transfer in Data-Constrained Environments

To improve LLM performance for low-resource languages, cross-lingual knowledge transfer from high-resource languages (primarily English) is essential. Existing methods require large amounts of parallel data, translation systems, or additional training stages, making them difficult to apply to low-resource languages.

To address these issues, the Apple research team proposed a data-level intervention technique called LINK. LINK works by randomly selecting words from the English portion of pre-training data and replacing them with their corresponding words in the target language using a bilingual lexicon.

This method is applicable with only a bilingual lexicon, without requiring additional model training or complex pipelines, and incurs almost no cost. Evaluation results across 8 languages and 5 model scales showed a noticeable improvement in downstream task performance for the target languages.

In particular, it was confirmed that the training time required to achieve equivalent performance was reduced by up to 2x. This is expected to significantly reduce the cost of developing AI models for languages with data constraints.