Understanding Word Error Rate (WER)

Word Error Rate (WER, Word Error Rate) is the standard metric that determines the accuracy of speech recognition (ASR) tools. A lower WER value means a more accurate transcription result, while a higher value indicates that information was recorded incorrectly.

WER is calculated based on three types of errors:

• Substitution: When a word is replaced with an incorrect different word
• Deletion: When a word is completely omitted
• Insertion: When a word that was not spoken is added

The calculation formula is WER = (S + D + I) / N, where N is the total number of words in the reference text. In practice, a WER of under 5% is considered a benchmark for excellent performance, but fields such as healthcare and law require an even lower error rate.

WER provides a basis for objectively comparing performance between models. According to a recent study by Artificial Analysis, ElevenLabs's Scribe v2 recorded a WER of 1.5% on the AA-AgentTalk dataset, showing industry-leading accuracy.

However, WER has the limitation of treating all errors equally, so it does not perfectly reflect contextual meaning. To address this, new metrics such as SWER (Semantic Word Error Rate), which measures whether the meaning of an utterance is preserved, are also emerging.