vLLM v0.28.0 Release: Kimi-K3 and DeepSeek V4 Optimizations, Significant Inference Performance Improvements

The vLLM v0.28.0 release includes 584 commits from 270 contributors, focusing on performance optimizations and infrastructure improvements for major LLM models.

Kimi-K3 and DeepSeek V4 Optimizations

Comprehensive stack-wide optimizations were implemented for Kimi-K3. These include Decode Context Parallel (DCP) support, FlashKDA decode and prefill kernel fusion, and SiTU activation support for MegaMoE. Additionally, adaptive speculative decoding token budgets improve DSpark TTFT (Time to First Token) by approximately 60%, and shared expert sharding saves approximately 17 GiB of memory per GPU. For DeepSeek V4, Sparse MLA is now supported end-to-end across standard decode, MTP, and DSpark speculative decoding, with enhanced AMD Quark NVFP4 support and ROCm compatibility.

Inference and Model Runner Improvements

Speculative decoding techniques have evolved with the introduction of DFlash2 (including local convolution and candidate selector) and DSpark reliability-based verification. Model Runner V2 has reached maturity with support for E/P disaggregation, weight offloading, and multi-layer MTP KV cache. Furthermore, disk offloading support and a standard CPU layout for parallelism-independent offloading were added for hierarchical KV cache offloading.

Default Configuration Changes and Others

The default max_num_batched_tokens has been increased from 8192 to 16384, and prefix caching is now enabled by default for Mamba models. The Rust frontend and gRPC-related features have also been expanded to include an independent renderer, multimodal image inference support, and explicit data parallel rank routing. Some compatibility changes exist, such as bitsandbytes support being moved to an external plugin.