NVIDIA Accelerates Google DeepMind's DiffusionGemma for Local AI

Google DeepMind has unveiled DiffusionGemma, an experimental open model that dramatically increases text generation speed. NVIDIA has optimized this model to run even faster on GeForce RTX GPU, RTX PRO, and DGX Spark systems.

Unlike the sequential (autoregressive) approach of existing LLMs, DiffusionGemma works like an image generation model, starting from noise and refining an entire block of text all at once. This allows it to generate up to 256 tokens in parallel in a single step, significantly reducing latency in single-user workloads.

Key features include:

• Built on Gemma 4: Uses a Mixture-of-Experts (MoE) architecture with 26 billion parameters, activating only 3.8 billion parameters per step.
• Performance boost: Delivers up to 4x faster generation speed compared to existing models in the same single-user environment.
• Optimized for local execution: An open-weight model released under the Apache 2.0 license, running entirely locally on RTX and DGX Spark without cloud costs.

In NVIDIA GPU environments, DiffusionGemma shows overwhelming performance. It generates 1,000 tokens per second on a single NVIDIA H100 GPU, and supports high-speed inference of up to 800 tokens per second on DGX Station. It is now available for immediate testing and prototyping through Hugging Face Transformers, vLLM, and Unsloth.