AI BriefingReaDy: Speculative Decoding Method Achieves 3.16x Speedup Using Hybrid Linear Attention Recurrent States
·2026.10.23 15:05
Key point
The method outperforms DSpark by improving average throughput by 7.97% and acceptance length by 8.50% across seven benchmarks.
Details
- ReaDy is the first speculative decoding method to leverage Hybrid Linear Attention (HLA) recurrent states for position-specific context retrieval during drafting.
- Achieves an average 3.16x decoding speedup over autoregressive decoding with Qwen3.8-27B on a single NVIDIA B200 GPU.
- Outperforms DSpark on all seven tested benchmarks, improving average throughput by 7.97% and acceptance length by 8.50%.
- Utilizes a lightweight state reader with learned queries to retrieve context from committed recurrent states, integrated via residual updates.
- Employs low-rank state reads and fused operators to minimize additional computation and I/O overhead.
- Built on DSpark's parallel backbone and sequential prediction head with a two-stage adaptation strategy.
- Code and model weights are available on GitHub and Hugging Face.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
llama.cpp Contributor Achieves Up to 140x Faster Prompt Lookup Drafting with New Optimizations
Speculative Decoding Speeds Up LLMs by Shifting GPU Work from Memory-Bound to Compute-Bound