vLLM Bottleneck Diagnostic Tool 'Profile' Released

Profile v2.2, an inference tuning tool that analyzes vLLM's real-time metrics, compares them against the GPU's performance limits (roofline ceiling), identifies bottlenecks, and suggests solutions, has been released.

Testing the Qwen3.8-27B model in a single H100 80GB environment achieved the following performance improvements through three iterations of tuning:

• Throughput: 257 → 490 tok/s
• Time to First Token (TTFT): 1.9s → 539ms (p95: 4.2s → 1.9s)
• Cost and Energy Efficiency: $3.23 → $1.69 per 1M output tokens, 2.39 → 1.00 J per token

In particular, it provides the ability to automatically detect KV cache pressure issues that occur under increased agent loads, and immediately restore performance by suggesting the application of fp8 KV cache and context length adjustments.

It currently supports vLLM, with multi-GPU and Tensor Parallelism (TP) support planned for addition in the future.