vLLM v0.28.0 Released with Major Optimizations for Kimi-K3 and DeepSeek V4

The vLLM v0.28.0 release has been published. This update includes 584 commits from 270 contributors, focusing particularly on full-stack optimizations for the Kimi-K3 and DeepSeek V4 models.

Kimi-K3 Performance Optimization

Several key technologies were introduced to improve inference speed and memory efficiency for the Kimi-K3 model. Support for Decode Context Parallel (DCP) was added, along with fused FlashKDA decoding and prefill kernels. Additionally, SiTU activation support for MegaMoE and GEMM-RS for sequence parallelism were included. The release also features fused all-gathers that deliver 1.5–3x speedups at the kernel level and an adaptive speculative token budget feature that improves DSpark TTFT by approximately 60%. A shared-expert sharding option that saves about 17 GiB of memory per GPU is also provided, along with added support for the V2 model runner in ROCm environments.

DeepSeek V4 and Speculative Decoding Improvements

For DeepSeek V4, support has been completed for sparse MLA to work end-to-end in plain decode, MTP, and DSpark speculative decoding. AMD Quark NVFP4 support and reasoning-effort prompt mapping were added, while work on sparse top-k metadata kernel optimization and reducing eager CUDA graph areas was undertaken. Support for ROCm's gfx11 and gfx950 architectures was also expanded.

In the area of speculative decoding, DFlash2 introduced local convolution and a candidate selector, while DSpark supports confidence-scheduled verification. Improvements were made to automatically enable async scheduling for draft models. Model Runner V2 supports E/P/D disaggregation and is maturing.