DFlash 2: Maintaining Parallel Drafting

Inco AI released DFlash 2, a speculative decoding technology. While the original DFlash adopted a method of predicting entire blocks in parallel, DFlash 2 demonstrates efficiency by increasing the output per verification pass by more than 20% while increasing latency by only about 1%.

This technology is already integrated into major inference engines such as SGLang, vLLM, TensorRT-LLM, and llama.cpp, with a maximum throughput improvement of 15x measured on NVIDIA Blackwell GPUs. DFlash models have been downloaded over 3.5 million times on Hugging Face.

The core of DFlash 2 lies in more accurately selecting the correct answer from a list of candidate tokens. Data shows that the probability of the correct answer being included among the top 16 candidates reaches 99.5%, but the probability of the top candidate being correct is only 85.4%. DFlash 2 leverages the potential of such candidate lists to maintain accuracy to the end of the block, achieving a throughput improvement of 2.7–3.4x compared to autoregressive decoding for the Qwen3.8-27B model.