AI Briefing#speculative-decoding
The latest AI and developer news about #speculative-decoding, with the original source and a short summary.
The latest AI and developer news about #speculative-decoding, with the original source and a short summary.
DeepSeek has released DSpark, a speculative decoding framework that dramatically improves LLM response speed, under the MIT license.
DeepSeek released DeepSpec, a tool for training and evaluating models for Speculative Decoding.
DSpark unveiled a speculative decoding framework that combines semi-autoregressive generation with confidence scheduling to solve the sharp acceptance rate decline problem of parallel drafters.
A JetSpec research paper has been released, dramatically increasing LLM inference speed through parallel tree drafting technology.
Covers how to dramatically boost inference speed for coding agents by leveraging Speculative Decoding and kernel optimization tailored to their task characteristics.
The Shard project dramatically improved distributed inference speed for large-scale models by leveraging speculative decoding and other techniques in a WAN environment.
Samsung R&D Institute has proposed PIX-TAB, a highly efficient pixel-precise table structure recognition technology capable of on-device execution.
DFlash technology, developed jointly by Z Lab and Modal, achieved LLM inference performance far superior to existing MTP methods.
EAGLE3, an inference acceleration technology, has been merged into llama.cpp, improving inference efficiency.
Xiaomi has unveiled MiMo-V2.5-Pro-UltraSpeed, which generates 1,000 tokens per second by applying FP4 quantization and DFlash technology.
MTP heads for Gemma 4 QAT models have been released, along with news of a fix for a llama.cpp parallel execution bug.
Domino, a technique that separates modeling from drafting to maximize the efficiency of Speculative Decoding, has been unveiled.
llama.cpp has improved speculative decoding speed by about 45% through a multi-column MMVQ port for Intel Arc GPUs.
This introduces Speculative KV coding, a technique that uses a predictor model to losslessly compress an LLM's KV cache by up to 4x.
A Qwen3.6 GGUF model has been released that integrates an MTP head, enabling self-speculative decoding without a separate draft model.
By applying MTP to the Qwen3.6-35B-A3B model, a decoding speed improvement of about 1.49x was achieved on llama-server.
LM Studio added support for MTP speculative decoding in 0.4.14 Beta.
Bug fixes related to MTP and speculative decoding improvements have been merged into llama.cpp.
MTP speculative decoding has been added to llama.cpp, making Qwen3.6 27B up to 2.44x faster.
The llama.cpp b9180 release adds MTP support and partial rollback.
llama.cpp added MTP support, and Qwen3.6 MTP GGUF models were released.
Orthrus claims up to a 7.8x speedup on Qwen3-8B inference while keeping the backbone frozen.
The **attention drift** phenomenon in speculative decoding drafters was identified and its performance was improved.
llama.cpp is undergoing updates to resolve conflicts between MTP and multimodal features and support their integration.