AI Briefinglibstrix plugin delivers up to 31x faster prefill for vLLM on Strix Halo APUs
·2026.10.29 15:56
Key point
The open-source plugin targets gfx1151 hardware to optimize MoE GEMMs and attention layers, reducing 256k prompt prefill time from over two hours to four minutes.
Details
- libstrix is a vLLM plugin containing custom HIP kernels for the gfx1151 architecture (Radeon 8060S), addressing performance gaps in stock ROCm kernels.
- Prefill performance improves significantly, with 256k prompt processing for Qwen3.6-35B-A3B dropping from 7,568 seconds to 241 seconds (a ~31x speedup), while 64k prefill speeds increase from 260 to 2,617 tok/s.
- The plugin integrates via vLLM's standard plugin system without forking, activating only on ROCm + gfx1151 and falling back to stock kernels for mismatched layer shapes.
- Accuracy remains consistent with stock vLLM for Qwen models (KLD 0.001-0.006), while Gemma-4-26B shows higher drift (KLD ~0.2) due to sensitivity to bf16 rounding.
- Decode speeds show mixed results, with gains limited for bf16 and int4 models where stock kernels are not the bottleneck, and MXFP4 gains attributed to replacing a slow emulation path.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
JetBrains releases ThinkRail, an open-source agentic IDE built on git worktrees and spec graphsNvidia OpenShell: Secure Runtime for Autonomous AI Agent Fleets