ROCm Turnabout?

On a Ryzen AI MAX+ 395-based Bosgame M5 with 128GB LPDDR5x, running llama.cpp via Proxmox VE 9.1 LXC GPU passthrough, Vulkan (b8816) and ROCm + rocWMMA (b8823) were compared.

• Measurements were taken in a cold state after reboot, with tuned accelerator-performance enabled.
• Common options were -ngl 999 -fa 1 --mmap 0 -b 4096 -ub 512 -t 8.

pp512 performance results varied by model.

• Gemma 4 26B-A4B: Vulkan ~1305 t/s vs ROCm 1043 t/s, with Vulkan 25% faster.
• Qwen3.5 35B-A3B: ROCm led by 7% on Q4_K_M and 5% on Q8_0.
• On MXFP4_MOE, ROCm 994 t/s vs Vulkan 693 t/s, with ROCm 43% ahead.
• GPT-OSS 120B: ROCm 651 t/s was 39% faster than Vulkan's 468 t/s.
• Some dense/lightweight models like Hermes 4.3 36B and MiniMax M2.7 favored Vulkan or were roughly even.

For tg128 performance, the gaps were mostly small.

• Gemma 4 26B-A4B: Vulkan 54 t/s, ROCm 48 t/s
• Qwen3.5 35B-A3B Q8_0: Vulkan 53 t/s, ROCm 45 t/s
• GPT-OSS 120B: ROCm 37.5 t/s, Vulkan 34 t/s
• Hermes 4.3 36B: both at 10 t/s, interpreted as a bandwidth limitation

The key takeaway is that for standard quantizations, Vulkan and ROCm are nearly the same, but for MXFP4-specific models, ROCm has a significant advantage. Conversely, Vulkan gained an additional about 9.3% with tuned accelerator-performance applied, while ROCm showed virtually no change.

In conclusion, this real-world benchmark shows that for MXFP4 models like GPT-OSS, ROCm is essentially required, while for most other cases, Vulkan + tuned is on par or better.

Additionally, LXC passthrough worked with the default Proxmox kernel (6.17) + amdgpu module, and since ROCm 7.2.2 --no-dkms could be used inside the privileged container, there was no need to install amdgpu-dkms on the Proxmox host.