H100 Qwen·Gemma Performance Comparison

8 models were compared using a single H100 80GB, vLLM 0.19.1, and vllm bench serve.

• Conditions: 100 prompts, 128 input tokens / 128 output tokens
• Concurrency: 1, 4, 8, 16 requests
• Metrics: throughput (TPS) and TTFT (time to first token)

In the results, Gemma 4 E2B-it was the strongest. At 16 concurrent requests, it recorded 3,180 TPS, while Gemma 4 31B dense managed only 226 TPS. TTFT also diverged significantly, at 55ms versus 4.1 seconds.

The effect of FP8 quantization was also clear.

• Qwen 3.6 35B MoE FP8 was 73% faster than BF16.
• The dense Qwen 27B with the same method improved by only 27%.
• The interpretation is that since expert weight movement is the bottleneck for MoE models, FP8 benefits not just memory savings but also actual speed gains.

Additionally, Gemma 4 31B dense was usable at low concurrency, but latency sharply worsened at 4 or more concurrent users. The conclusion is that for serving a 30B-class model on a single H100, MoE is more suitable than dense.