DiffusionGemma: 4x Faster Text Generation

Google has announced DiffusionGemma, an experimental open model that maximizes generation speed by leveraging text diffusion technology. Released under the Apache 2.0 license, this model adopts a 26B Mixture of Experts (MoE) architecture, overcoming the limitations of token-by-token sequential processing inherent in existing Autoregressive approaches.

DiffusionGemma achieves up to 4x faster speed on GPUs by generating entire blocks of text simultaneously. Key features include:

• Ultra-fast Inference: Generates over 1,000 tokens per second on NVIDIA H100, and outputs over 700 tokens even on RTX 5090.
• Efficient Hardware Utilization: Of the total 26B parameters, only 3.8B parameters are activated during inference, allowing smooth operation on consumer GPUs with 18GB VRAM when quantized.
• Bi-directional Attention: Generates 256 tokens in parallel in a single forward pass, making it advantageous for code infilling or mathematical graph generation.
• Intelligent Self-correction: Undergoes an iterative refinement process that evaluates entire text blocks at once to correct errors in real time.

However, as this is an experimental model optimized for speed and parallel generation, using the standard Gemma 4 model is recommended when top-level output quality is required.