Qwen3.6 27B on Dual RTX 5060 Ti, 60 tok/s

Performance was measured running sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP on a combination of dual RTX 5060 Ti 16GB, CUDA 13 / Torch 2.11 nightly, and vLLM nightly 0.19.2rc1.dev.

• At 8K context, MTP n=1 recorded 50-52 tok/s, and n=3 recorded 62-66 tok/s.
• At 32K context, it maintained 59-66 tok/s.
• Startup and operation with 204,800 context were confirmed, and after 168k-token prefill, VRAM per GPU was 15.65GiB.
• Idle VRAM was 14.45GiB per GPU, and a 168k-token needle/retrieval smoke test passed in about 256 seconds.
• Prompts+outputs exceeding the limit were correctly rejected.

The run was configured with --tensor-parallel-size 2, --max-model-len 204800, --kv-cache-dtype fp8, --quantization modelopt, and --attention-backend TRITON_ATTN. gpu_memory_utilization=0.94 failed KV allocation, and only 0.95 passed.

Thinking mode also works, but if max_tokens is too low it fails to produce a final output, so 1024+ is required, and 4096-8192 was safer for actual inference. Startup took several minutes due to compilation and autotuning, and even when OOM fallback logs from the FlashInfer autotuner appeared, the server still started normally.