Apple Researchers Propose Stabilized Semi-Supervised Federated ASR with Online Pseudo-Labels

Apple researchers have developed a practical recipe for Semi-Supervised Federated Learning (SSFL) in Automatic Speech Recognition (ASR), addressing the fragility of pseudo-labels that typically causes training divergence. The study identifies two coupled design axes—teacher selection and server-side anchoring—as critical for closing the performance gap between semi-supervised and fully-supervised federated learning.

Teacher and Anchor Dynamics

The research demonstrates that while a per-client online teacher (using each client's evolving model) can diverge on its own, it matches or outperforms a static broadcast global teacher once training is stabilized. This advantage is decisive in in-domain scenarios and competitive under domain shift. As the labeled seed dataset grows, a transitioning teacher strategy—switching from global to online at a specific round—matches or beats both static approaches.

Crucially, the anchor axis requires the server to continue training on labeled data between rounds. This interleaving prevents the online teacher from drifting and governs convergence more effectively than the initial seed model alone. The two axes are inseparable: aggressive teacher choices only yield benefits when the anchor stabilizes training, a process highly sensitive to data augmentation and batch size settings that control input and gradient noise.

Performance and Guidelines

The proposed method improves over the strongest prior SSFL techniques on 9 of 11 tested pairs. It achieves an average accuracy improvement of 20.8% in-domain and 10.0% cross-domain, significantly narrowing the gap to fully-supervised federated learning. The findings provide specific guidelines for SSFL in ASR, noting that the required level of stabilization is domain-dependent, governed by the dispersion of the seed data and its overlap with client data.