RTX6k Qwen3.5 Benchmark Updated
Key point
The latest llama.cpp build significantly improved the RTX6k Qwen3.5 benchmark.
Details
After upgrading llama.cpp from b8198 -> d05fe1d, Qwen3.5-122B-A10B (MXFP4_MOE) was re-measured on RTX6k (Server, 450W) with the same model, same hardware, and same flags. CUDA toolkit remained at 13.0, and the new build includes hybrid SSM/MoE speculative decode, native NVFP4 MMQ for SM120, server prompt cache, and MoE/MXFP4 kernel improvements from the last 2 months.
Measurements were performed by sending synthetic prompts to the v1/completions endpoint of a running server. cache_prompt:false, n_predict=128-256, temperature=0, and chat_template_kwargs.enable_thinking=false were used, and speed was parsed from the server response. The production setting was changed from --parallel 4 to --parallel 2.
The key results are as follows.
- pp512: 2,188 -> 3,176 t/s, +45%
- tg single-stream: 80.0 -> 106.6 t/s, +33%
- tg per-req @ c=2: 55.7 -> 89.3 t/s, +60%
- Total tg @ c=2: 111.4 -> 178.6 t/s, +60%
- pp 8K / 32K / 65K: +69% / +65% / +59%
- tg 8K / 32K / 65K: +35% / +35% / +28%
- TTFT 8K / 32K / 65K: 2,780 / 10,780 / 23,161 ms -> 1,877 / 7,955 / 17,737 ms
The improvement in prompt processing was larger than in token generation, and TG dropped by about -13% from 0->65K, maintaining a slope similar to the previous -10%.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.