AI Briefinglibstrix Docker image enables Qwen3.8-Flash-Next on Strix Halo with 40+ tok/s decode
·2026.10.31 06:10
Key point
New Docker image bundles vLLM fork and ROCm 10.1 for Strix Halo, achieving 40-43 tok/s decode for Qwen3.8-Flash-Next.
Details
- One-command Docker image (ghcr.io/asf0/strix-vllm:latest) bundles vLLM fork, libstrix plugin, and ROCm 10.1 for Strix Halo (gfx1151), requiring only the amdgpu driver on the host.
- Qwen3.8-Flash-Next achieves 40-43 tok/s decode and 1,539-1,617 tok/s prefill with MTP 2, using a custom vLLM fork to stream the 102 GB n-gram table from disk.
- Performance benchmarks show Qwen3.8-Flash-Next AWQ int4 reaching 24.7-26.1 tok/s decode at 2k-64k context lengths, with accuracy KLD of 0.008-0.019 against stock vLLM.
- Memory overhead table provided for 11 model configurations, ranging from 15.6 GiB (gemma-4-26B-A4B AWQ) to 79.5 GiB (Qwen3.8-Flash-Next AWQ) for weights and KV cache.
- Recent update limits int8 decode copies to Qwen3.8-Flash-Next (capped at 12 GB) to prevent memory exhaustion on other models like Qwen3.8-27B bf16.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Ubicloud Achieves 100% Tool-Call Accuracy in vLLM via Constrained DecodingvLLM offers PyTorch-native TPU backend for Google Cloud
Figma's unreleased editing model, post-trained on Qwen 3.8 27B, beats top third-party model on internal evals