AI BriefingApple Researchers Propose Lightweight Code-Switching Detection Using Speech Foundation Models
Key point
The method achieves a mean F1 score of 0.798 across 14 language pairs in CS-FLEURS using the Whisper-small encoder.
Details
Apple researchers have developed a lightweight method to detect code-switching (CS) directly from speech, addressing the underrepresentation of multilingual alternation in pretraining corpora. The approach leverages frame-level language information in pretrained speech encoders to identify whether both languages are present within a single utterance, bypassing the need for accurate multilingual transcriptions.
Methodology
The technique identifies language-selective hidden-state dimensions using monolingual recordings and aggregates their frame-level activations to generate a CS prediction. A small CS set is used to tune the decision rule, allowing the system to function without heavy computational overhead.
Performance Results
When tested on the CS-FLEURS dataset, the Whisper-small encoder achieved a mean F1 score of 0.798 across 14 language pairs. This performance surpasses baseline models, including:
- A CNN baseline with an F1 of 0.776
- A HuBERT linear baseline trained for monolingual language identification with an F1 of 0.751
The findings indicate that pretrained speech encoders already contain sufficient information to identify CS utterances, offering a practical tool for constructing more representative multilingual datasets.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.