Red Hat Publishes Performance Comparison Commentary on vLLM vs SGLang

Red Hat presented selection criteria for different workloads based on the core differences in KV cache management between vLLM and SGLang, along with benchmark data. Both engines have established themselves as standards for serving open-weight models, but they differ in memory efficiency and latency optimization strategies.

Differences in KV Cache Management

• vLLM (PagedAttention): Splits the KV cache into fixed-size blocks and allocates them non-contiguously, similar to OS virtual memory. This resolves 60–80% waste caused by memory fragmentation in existing systems and increases the number of concurrent requests.
• SGLang (RadixAttention): Stores the cache in a Radix Tree keyed by token prefixes. It merges and reuses common prefix paths, and improves cache hit rates with Cache-aware Scheduling.

Benchmark Results and Interpretation

According to data from Modal's LLM Engineer's Almanac (based on Llama 3.1 8B on H100), the performance of the two engines is similar under default settings.

• RAG/Summarization (Input 1024/Output 128): vLLM at 17.08 req/s and SGLang at 16.88 req/s, with vLLM holding a slight edge.
• Code Generation (Input 128/Output 1024): SGLang recorded a throughput of 6.50 req/s, approximately 22% higher than vLLM (5.34 req/s).
• Long Chat (Input 2048/Output 2048): vLLM at 1.23 req/s and SGLang at 1.25 req/s, showing no significant difference.

Workload-Specific Selection Guide

• Offline Batch Processing: vLLM recommended.
• Latency-Sensitive Online Serving: SGLang recommended, with Tensor Parallelism and EAGLE-3 Speculative Decoding applied.
• Non-NVIDIA Hardware: vLLM has a longer history of support, but SGLang also has documentation for TPU/Intel/CPU/Apple Silicon.
• Agents with High Prefix Reuse: This area is not measured in public benchmarks, so self-measurement using tools like GuideLLM is necessary based on actual traffic patterns.