Qwen3.6-27B vLLM Docker, 118 TPS

A vLLM Docker configuration was released that serves Qwen3.6-27B on 2x RTX 3090 GPUs using Lorbus AutoRound INT4 quantization and MTP speculative decoding.

• Performance: Measured 118.13 tok/s on coding workloads and 88.98 tok/s on prose.
• Configuration: Uses a combination of TP=2, 200K context, fp8 KV cache, chunked prefill, and num_speculative_tokens=3.
• Deployment method: The model is downloaded from Hugging Face at runtime and stored on a host volume, avoiding re-downloads on container upgrades.
• Compatibility: Provides an OpenAI-compatible API, MoonViT-based image input, automatic GPU detection, and single/multi-GPU execution examples.
• Operational notes: Works on PCIe setups as well, and states that NVLink is not required.

However, it notes that prefix caching is experimental with Qwen3.6's hybrid attention, and spec-decode ignores min_p and logit_bias. It also states that CUDA graph mode falls back to PIECEWISE under spec-decode.