Samsung Electronics Unveils 'MusicJudge', an AI Song Evaluation Framework Reflecting Musical Structure

Researchers from the Samsung Electronics R&D Institute India-Bangalore proposed MusicJudge, an automatic song evaluation framework that reflects musical structure, at Interspeech 2026. Existing evaluation methods either focused solely on acoustic quality or lyric transcription, or ignored musical context, resulting in limitations such as penalizing artistic freedoms like melisma or tempo flexibility, or missing lyric errors.

MusicJudge divides songs into musically meaningful blocks such as verse and chorus, simultaneously evaluating Content fidelity (lyric accuracy) and Musical fidelity (structural adherence of pitch/rhythm). The pipeline first separates vocals and accompaniment via Source separation, then performs transcription robust to song-specific vocal patterns using a Whisper model fine-tuned with MG-LoRA. It subsequently undergoes multi-signal-based block detection and musical pitch/rhythm scoring to produce final scores.

Experimental results showed a Spearman correlation of 0.683 with human expert rankings on datasets such as SwaraLyrics, demonstrating performance improvements of 31.9% over lyrics-only and 38.0% over music-only approaches. Notably, the application of MG-LoRA improved song transcription accuracy by approximately 29.87% compared to the second-ranked system, maintaining high accuracy even in complex vocal passages such as gamakas or portamento.

This framework estimates the key from the accompaniment to avoid penalizing the singer's transposition, and treats song structure as a latent variable to fairly evaluate order changes such as live covers. While currently limited to solo performances, it can be expanded to duets or choirs in the future, and could be utilized as a vocal training aid or for preliminary judging in music competitions.