MMFlash Achieves Up to 3.58x Lossless Speedup for Vision-Language Models via Adaptive Speculative Decoding
·2026.11.16 19:48
Key point
The framework uses visual-dependence-aware training and sparse attention to outperform EAGLE-3 and DFlash on multimodal benchmarks.
Details
- MMFlash introduces a speculative decoding framework for vision-language models (VLMs) that achieves lossless speedups of up to 3.58x across various multimodal benchmarks.
- The method outperforms existing baselines EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively, by adaptively upweighting visually dependent tokens that the drafter predicts poorly.
- A token-level visual dependence metric based on KL divergence reveals that acceptance rates drop from 46% in the lowest visual-dependence quartile to 22% in the highest, a disparity that persists after multimodal training.
- For long video inputs, MMFlash employs sparse attention to reduce redundant computation, maintaining efficiency on 40k-token video prompts while keeping the drafter cost independent of the number of visual tokens.
- Released drafts for Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B, and Gemma 4 E4B-it are available on Hugging Face and run with stock SGLang and vLLM without patches.
- Specific performance gains include 3.0x speedup on Qwen3.5-4B/9B and up to 4.7x on Qwen3.8-27B using a single H200 GPU with batch size 1 and greedy decoding.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.