AI BriefingQwen3.6-27B, 72 tok/s on Windows
Key point
On Windows 10 with native vLLM, Qwen3.6-27B hit 72 tok/s.
Details
Running Qwen3.6-27B on Windows 10 with patched native vLLM, executed without WSL or Docker, achieving 72 tok/s on an RTX 3090.
- Short prompt: 72 tok/s
- Long prompt (about 25k tokens): 64.5 tok/s
- 127k ctx single GPU: 53.4 tok/s
- PP=2 / 2×3090: 160k ctx
The deployment is provided as a portable ZIP in a GitHub release, requiring no admin rights, pip, or Python—just unzip and run start.bat. On first run, it installs the bundled vLLM wheel and dependencies into the embedded Python, and if needed, automatically downloads Lorbus AutoRound INT4 quantization from Hugging Face.
Testing was done on Windows 10 + 2×RTX 3090, and the author expects it to work on Ampere/Ada/Blackwell series cards. The OpenAI-compatible endpoint is http://127.0.0.1:5001/v1.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Qwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics