llama.cpp NVFP4 Comparison

On an RTX 5090, Ryzen 9 9950X3D, 128GB DDR5 5600 CL36, CUDA environment, Qwen3.6-27B-NVFP4 was compared between llama.cpp b8966 (non-native) and b8967 (native). The llama-bench label showed qwen35 27B NVFP4, but the actual model tested was Qwen3.6-27B-NVFP4.

• Model configuration: 17.50 GiB, 26.90B params, ngl=999, fa=1
• Prompt processing performance was consistently ahead with b8967, with improvements ranging from +42.6%~+68.3%.
• The average prompt processing improvement was about 57%.
• pp512 rose from 3295.10 → 5546.93 t/s, and pp2048 rose from 3373.30 → 5594.58 t/s, showing a significant speedup in prefill.
• pp512 @ d32768 also recorded 2514.70 → 3586.58 t/s, and pp2048 @ d32768 recorded 2479.39 → 3560.58 t/s.
• On the other hand, token generation speed remained virtually unchanged, so the perceived gains are largest with long prompts, RAG, document analysis, and code-heavy inputs.

In other words, native NVFP4 support significantly boosted llama.cpp's input processing stage on Blackwell, but the generation stage remained nearly the same.