AI Briefingllama.cpp -sm tensor improvement
·2026.04.30 02:50
Key point
With a 2x RTX 3090 Ti setup, applying `-sm tensor` increased llama-bench throughput.
Details
llama-bench results targeting Qwen3.6-27B-UD-Q4_K_XL.gguf were shared.
- On a single RTX 3090 Ti setup, with
-fa 1, it recorded pp512 1580.12 t/s and tg128 44.43 t/s. - On a 2x RTX 3090 Ti setup, with
CUDA_VISIBLE_DEVICES=0,1and-fa 1 -sm tensorcombined, it went up to pp512 2047.28 t/s and tg128 58.83 t/s. - Both results used the same build 665abc609 (8951).
The post notes that just a few weeks ago, llama-bench didn't work on the -sm tensor path, suggesting that this multi-GPU path has recently been improved.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
