4060 Ti Monster Speed

On a Windows 11 setup with an i7-13700F, 64GB RAM, and RTX 4060 Ti 16GB, llama.cpp was tuned to stably run Qwen3.5-35B-A3B-UD-Q4_K_L GGUF at 64k context.

The core preset is ctx-size=65536, threads=6, threads-batch=8, n-cpu-moe=11, batch-size=1024, ubatch-size=512, parallel=2, kv-unified=true, ngl=99, fa=on, ctk/ctv=q8_0.

It was launched with llama-server.exe --models-preset models.ini --models-max 1 --host 0.0.0.0 --webui-mcp-proxy --port 8080, and even with Docker Desktop running, it achieved 40-60 tok/s in real-world use.

Example log values are around 56.41 tok/s, 46.84 tok/s, 44.97 tok/s, 41.21 tok/s, and 42.71 tok/s.

The author notes that rather than just chasing the highest peak number, it was more important to look at the actual runtime shape via values like n_parallel, kv_unified, n_ctx_seq, n_ctx_slot, n_batch, n_ubatch, and concludes that reducing VRAM pressure had a bigger effect on perceived performance.