Apple's New SpeechAnalyzer API Benchmarked Against Whisper and Previous Models

Apple's new on-device speech engine SpeechAnalyzer outperformed both the existing SFSpeechRecognizer and OpenAI's Whisper models in LibriSpeech benchmark tests.

Key Benchmark Results (based on LibriSpeech):

• Accuracy (WER): SpeechAnalyzer recorded a word error rate (WER) of 2.12% on clean speech and 4.56% on noisy speech, showing higher accuracy than Whisper Small (3.74% / 7.95%).
• Speed: SpeechAnalyzer showed approximately 3x faster processing speed compared to Whisper Small.
• Comparative advantage: The existing SFSpeechRecognizer recorded the lowest performance with a WER of 9.02% on clean speech.

Conclusion and Implications:

• For English speech recognition, when using Apple hardware (iOS/macOS), SpeechAnalyzer has the advantage over Whisper in both accuracy and speed.
• However, Whisper has the advantage of versatility in that it works in environments outside the Apple platform and supports far more languages.
• This result is a case showing just how powerful on-device AI model optimization can be when combined with hardware acceleration.