I-DLM - Introspective Diffusion Language Models

I-DLM is a case where a diffusion-based language model aims to achieve AR (Autoregressive) model-level quality and parallel generation speed at the same time.

The core is Introspective Strided Decoding (ISD), which performs new token generation and verification of previous tokens together in a single forward pass. To reduce the weaknesses of existing DLMs — lack of self-consistency, inefficient computation, and serving infrastructure mismatch — it applies introspective-consistency training.

In terms of performance, I-DLM-8B showed improvements of +26 points on AIME-24 and +15 points on LiveCodeBench-v6 compared to LLaDA-2.1-mini (16B), even with half the parameters, and outperformed existing DLMs across 15 benchmarks overall. The figures are summarized as ARC-C 96.8, MMLU-Pro 79.7, GPQA-D 62.1, AIME-24 83.3, MATH-500 97.6, HumanEval 96.3, etc.

In terms of serving and speed, it achieved 2.9-4.1x throughput compared to LLaDA-2.1-mini and SDAR in the batch 1-64 range, and is directly compatible with SGLang infrastructure. It also presents lossless R-ISD using Gated LoRA, which enables acceleration while maintaining bit-level identity with the base AR output.

The models released are I-DLM-8B, I-DLM-32B, and I-DLM-8B-LoRA, along with documentation and scripts for installation, training, inference, serving, and benchmark reproduction.