AI Briefing
KOSign in

vLLM v0.33.0 Scheduler Fix Boosts DeepSeek DSpark Throughput by 2.47x

·2026.11.09 03:39

Key point

Removing unnecessary draft-token reservations allows larger prefill chunks and more efficient CUDA graph execution, raising output throughput from 1,373 to 3,390 tok/s.

Details

  • vLLM v0.33.0 removes unnecessary draft-token reservations for DSpark and DFlash, separating draft-buffer capacity constraints from the target model's scheduling budget.
  • Output throughput for DeepSeek-V4.1-Flash with DSpark (K=5) increased from 1,373 to 3,390 tok/s (+147%) on a single GB300 with TP4.
  • Median Time Per Output Token (TPOT) dropped from 29.2 ms to 15.7 ms, and median Time To First Token (TTFT) decreased from 93.6 s to 30.0 s.
  • The fix enables larger prefill chunks and reduces mixed prefill/decode iterations, allowing the system to utilize pure-decode CUDA graphs more effectively.
  • Observed maximum running requests increased from 52 to 64, while acceptance length remained stable at approximately 4.72 to 4.73.
  • Results are specific to the DeepSeek-V4.1-Flash DSpark benchmark configuration and do not establish a universal speedup; DFlash was not benchmarked end-to-end in this change.
deepseekspeculative-decodingvllmthroughputinference-optimizationscheduler
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

LlamaIndex BlogLlamaIndex Achieves 10x Inference Speedup for OCRia-1 with Custom Optimization StackTLDR AIDeepSeek Open-Sources DeepInfer Framework for V4.1 Flash InferenceRedditTencent Introduces DSpine: Parallel Speculative Decoding with Causal InjectionPyTorchKRModular LLM Inference Handbook, 57 pages released for freeRedditlibstrix plugin delivers up to 31x faster prefill for vLLM on Strix Halo APUsTLDR AIDecagon's AutoResearch Agent Automates Inference Stack Optimization, Achieving Up to 2.5x Throughput Gains