NVIDIA Releases Nemotron 3 Diarization

NVIDIA released Nemotron 3 Diarization, an open-weight model with 100M parameters, via the Hugging Face blog. This model identifies speaker timestamps in overlapping conversations to support speaker-specific transcription when combined with ASR, and supports up to 8 speakers.

Performance and Benchmarks

In the initial results of the Voice Arena Diarization-Bench, it ranked first among 12 systems with a DER of 14.72%. This represents an approximately 24% relative reduction in error rate compared to the second-place system (19.3%). It lowered the DIHARD III DER from 19.09% to 12.73% compared to the existing baseline and significantly improved batch processing throughput (RTxF). However, results may change depending on the final evaluation and statistical analysis of Voice Arena.

Architecture and Processing Method

It processes 16kHz mono audio and adopts the Sortformer approach to assign speaker channels in order of first appearance. It ensures label stability during streaming through AOSC and a FIFO queue, and outputs anonymous channel IDs and metadata rather than actual identities.

Throughput and Constraints

With a batch size of 32 in a torch.compile() environment, it recorded RTFx 15,113x at a 30.4-second configuration. At a 1.04-second configuration, it recorded RTFx 865x, lowering the DER from 19.60% to 13.18%. Argmax Pro SDK 3 supports this model and provides real-time speaker attribution for up to 8 speakers.