Orthrus-Qwen3-8B 7.8x Speedup

Orthrus inserts a trainable diffusion attention module into each layer of a frozen AR Transformer, proposing K=32 tokens in parallel at once, with the AR head verifying the longest matching prefix in a second pass.

The two heads share a single KV cache, operating without an external drafter, and the paper claims this structure maintains the same output distribution as the base model.

• Recorded up to 7.8x tokens/forward on Qwen3-8B.
• Reported roughly 6x wall-clock speedup on MATH-500.
• Trained only 16% of total parameters, using under 1B tokens and 8x H200 for 24 hours.
• Compared against Dream, Fast-dLLM-v2, SDAR, Mercury, and Gemini Diffusion, it explains that it matches Qwen3-8B accuracy without changing the backbone.
• Unlike speculative decoding methods such as EAGLE-3 and DFlash, there is no separate drafter, no TTFT penalty, and KV overhead is fixed at O(1), around 4.5 MiB.
• Average acceptance length on MATH-500 was 11.7, longer than DFlash (7.9) and EAGLE-3 (3.5).
• single-step denoising outperformed multi-step (6.35 vs. 3.53 TPF), and KL distillation achieved a higher acceptance rate than CE.

The limitations are also clear. It inherits the biases, hallucinations, and knowledge gaps of the frozen base model as-is, and evaluation is limited to Qwen3-only with greedy + rejection sampling.