AI BriefingPaiton vLLM plugin enables Qwen3.8 27B inference on single AMD Radeon AI PRO R9700
·2026.11.09 06:56
Key point
The final Qwen3.8 release for the Paiton plugin achieves 180 tok/s decode speed on a single 32 GB AMD card and improves SWE-bench Verified scores for 3-bit models.
Details
- Paiton, a vLLM plugin with custom kernels for AMD RDNA4, supports Qwen3.8 27B on a single Radeon AI PRO R9700 and Qwen3.8 Flash Next on two cards.
- Qwen3.8 27B 3-bit achieves 180 tok/s decode speed (1 request) and 489 tok/s (8 requests) on a single R9700, supporting 65K context or 262K in long mode.
- Retrained 3-bit scales for the 27B model increased SWE-bench Verified solved tasks from 284 to 323 out of 500, reducing agent loop errors from 89 to 67.
- Qwen3.8 Flash Next 3-bit maintains near-16-bit performance with 97.4% GSM8K and 94.5% HumanEval accuracy, while 4-bit session hibernation resumes 33K-90K token contexts in 0.7 to 1.3 seconds from NVMe.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Ubicloud Achieves 100% Tool-Call Accuracy in vLLM via Constrained DecodingvLLM offers PyTorch-native TPU backend for Google Cloud