NVIDIA Releases Nemotron-Labs-Sigma 8B and 3B Continuous Diffusion Language Models
·2026.11.17 03:06
Key point
The 8B and 3B models use a 16-D continuous token-embedding space to denoise text in parallel blocks, supporting up to 8,192 tokens of context.
Details
- NVIDIA releases Nemotron-Labs-Sigma 8B and 3B, continuous diffusion language models that generate text by denoising Gaussian noise in a 16-D continuous token-embedding space rather than discrete tokens.
- The 8B model achieves reasoning accuracy competitive with state-of-the-art masked diffusion LMs while enabling block-parallel decoding where 64-token blocks are refined simultaneously.
- Inference uses DDPM or DDIM samplers with self-conditioning and optional classifier-free guidance, implemented in plain PyTorch without custom kernels or flash-attention.
- The SFT variants are fine-tuned for instruction following and are ready for commercial or non-commercial use, supporting English text with a maximum context length of 8,192 tokens.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.