Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

Code-Switching, the phenomenon of switching languages within the same utterance, poses a significant challenge to Automatic Speech Recognition (ASR) systems due to the scarcity of training data. This study is the first to apply an iterative pseudo-labeling approach to code-switching ASR, demonstrating its effectiveness in improving model performance by leveraging unlabeled data.

The approach consists of three main stages. First, pseudo-labels are generated from a large-scale unlabeled corpus to construct a semi-supervised dataset. Next, two-stage bilingual model training is performed based on this dataset, involving pre-training followed by fine-tuning on supervised code-switching data. Finally, iterative refinement further enhances model accuracy in complex code-switching scenarios.

The system applying this method achieved a significant reduction in Mix Error Rate (MER) on the SEAME devman (6.35%) and devsge (8.29%) subsets. This represents a significant advancement in substantially improving the accuracy of code-switching ASR systems.