vllm releases

Details

Breaking Changes

Discontinued support for CUDA 12.9 and transitioned exclusively to CUDA 13.0.
Removed Medusa/MLP speculator and the legacy Ray compiled-graph executor.
NIXL 1.5+ versions are now a mandatory requirement.
Removed three environment variables, including VLLM_ROCM_AITER_MLA_DCP_VERIFY.
Removed the VLLM_ROCM_MONO_DECODE environment variable and replaced it with the --monokernel flag.
The --trust-request-chat-template option is now required for chat template processing.

Major New Features

Introduced the Rust Frontend to achieve feature parity with Python and eliminate long-text HTTP 500 errors.
Default transition to ROCm 10.0 (the previous 7.2 is available via the -rocm72 option).
New API endpoints: added /v1/decisions, late chunking embeddings, token embeddings, and derender mode.
New model support: Reflection Beam-1.0, AliceAI-Foundation-80B-A3B (512-expert MoE, 262K context), Recursive Flint, Step-5 Preview, Nemotron-Labs Dory1.0-4B, MiniCPM-OCR 1.0, and others.
Applied GLM-5.3-Flash FP8 KV cache by default and added Cohere2MoE dual stream support.
Enabled FA4 sparse MLA decode on the Blackwell architecture when query heads per rank are 16 or fewer.

Performance Improvements

NVIDIA GB300: Improved from 1373→3390 tok/s during DFlash/DSpark speculative decoding by removing slot reservation.
Kimi-K3: Achieved 1.5-2x speed improvement in TP8/16 environments.
DeepSeek-V4.1-Flash: +18.3% performance improvement at Batch 1, reducing step time by ~850us at 1M context.
AMD ROCm: Applied MonoKernel to improve MI300X/MI325X P50 by 2.32x, reduce MI355X TPOT by -16.6%, and expand RX 7900 XTX KV cache by 4.6x.
GLM/Gemma4/Qwen3/MiniMax M2: Reduced tool streaming latency from 2.1s → 11ms at a 12KB baseline.
B300: Improved TPOT by 1.2-3.0%, and enhanced GB200 long-prompt staged writes by 31-120x.
CPU Offload: Shortened GLM-5.3 startup time from 63.6s → 38.3s by supporting 32 shm files.
Achieved up to 8x speed improvement by skipping Masked logits search.

Deprecations & Removals

Security & Fixes

Introduced media byte budget limits and applied certifi>=2026.10.19.
Added logic to return a 503 status code for requests during generation.
Normalized beam search score length, handled failures for invalid fields, and returned 400 for empty tools arrays.

Details

Breaking Changes

Discontinued support for CUDA 12.9 and transitioned exclusively to CUDA 13.0.
Removed Medusa/MLP speculator and the legacy Ray compiled-graph executor.
NIXL 1.5+ versions are now a mandatory requirement.
Removed three environment variables, including VLLM_ROCM_AITER_MLA_DCP_VERIFY.
Removed the VLLM_ROCM_MONO_DECODE environment variable and replaced it with the --monokernel flag.
The --trust-request-chat-template option is now required for chat template processing.

Major New Features

Introduced the Rust Frontend to achieve feature parity with Python and eliminate long-text HTTP 500 errors.
Default transition to ROCm 10.0 (the previous 7.2 is available via the -rocm72 option).
New API endpoints: added /v1/decisions, late chunking embeddings, token embeddings, and derender mode.
New model support: Reflection Beam-1.0, AliceAI-Foundation-80B-A3B (512-expert MoE, 262K context), Recursive Flint, Step-5 Preview, Nemotron-Labs Dory1.0-4B, MiniCPM-OCR 1.0, and others.
Applied GLM-5.3-Flash FP8 KV cache by default and added Cohere2MoE dual stream support.
Enabled FA4 sparse MLA decode on the Blackwell architecture when query heads per rank are 16 or fewer.

Performance Improvements

NVIDIA GB300: Improved from 1373→3390 tok/s during DFlash/DSpark speculative decoding by removing slot reservation.
Kimi-K3: Achieved 1.5-2x speed improvement in TP8/16 environments.
DeepSeek-V4.1-Flash: +18.3% performance improvement at Batch 1, reducing step time by ~850us at 1M context.
AMD ROCm: Applied MonoKernel to improve MI300X/MI325X P50 by 2.32x, reduce MI355X TPOT by -16.6%, and expand RX 7900 XTX KV cache by 4.6x.
GLM/Gemma4/Qwen3/MiniMax M2: Reduced tool streaming latency from 2.1s → 11ms at a 12KB baseline.
B300: Improved TPOT by 1.2-3.0%, and enhanced GB200 long-prompt staged writes by 31-120x.
CPU Offload: Shortened GLM-5.3 startup time from 63.6s → 38.3s by supporting 32 shm files.
Achieved up to 8x speed improvement by skipping Masked logits search.

Deprecations & Removals

Security & Fixes

Introduced media byte budget limits and applied certifi>=2026.10.19.
Added logic to return a 503 status code for requests during generation.
Normalized beam search score length, handled failures for invalid fields, and returned 400 for empty tools arrays.

vllm