ORL Research Achieves 2,788 Tokens/s on RTX 5090 for Diffusion LLMs

Researchers introduced ORL (On-Policy Residual Learning), a method to accelerate diffusion language models (dLLMs) without compromising generation quality. By combining ORL with the custom inference engine dlmrt, the team achieved 2,788 tokens/s on a single RTX 5090 using LLaDA-2.1-mini with NVFP4 quantization. On a larger scale, DiffusionGemma 26B MoE reached 7,708 tokens/s on 8x B200 GPUs.

Core Methodology

ORL addresses the challenge of parallel token refinement in dLLMs through two key techniques:

• On-Policy Residual Learning: The model trains on fresh trajectories generated by the current model, replaying exact residual states encountered during inference to avoid distribution mismatches.
• Adjacent Entropy Contraction: This loss function encourages each denoising step to reduce uncertainty relative to the previous step, teaching the model to finalize more tokens per forward pass.

Performance Benchmarks

The study reports significant throughput improvements and competitive accuracy across several models:

• QwenDLM-4B: Achieved up to 6.9x higher end-to-end throughput and improved HumanEval pass@1 from 82.9% to 89.6%.
• LLaDA-2.1-mini: Required only 4.2M self-generated training tokens and 36 H100 GPU-hours for post-training, delivering up to 3x higher throughput.
• Comparison to Speculative Decoding: On a single H100, QwenDLM-4B + ORL reached 1,298 tokens/s, outperforming DFlash (746 tokens/s) and EAGLE3 (405 tokens/s) at comparable accuracy.

Infrastructure and Availability

The team developed dlmrt, an inference engine optimized for low-latency, single-request diffusion decoding. It captures the entire denoising loop in a CUDA graph and supports NVFP4 inference on Blackwell GPUs. The project is fully open-source, including the paper, training code, evaluation pipeline, and model checkpoints.