Qwen3.6 35B 8GB Setup

Shared a setup for running Qwen3.6-35B-A3B GGUF with llama-server in an environment with RTX 4060 Laptop 8GB VRAM and 96GB RAM.

• The purpose is not interactive chat but a coding subagent.
• The runtime values are -ngl 99, --n-cpu-moe 99, -c 50000, -np 1, -fa on, --cache-type-k q8_0, --cache-type-v turbo2, --no-mmap, --mlock, --ctx-checkpoints 1, --cache-ram 0, --jinja, --reasoning on, --reasoning-budget -1, -b 2048, -ub 2048.
• In PowerShell, LLAMA_SET_ROWS=1 and preserve_thinking: true from LLAMA_CHAT_TEMPLATE_KWARGS were used.

The actual cause of the blockage wasn't a crash but the fact that thinking was consuming the entire max_tokens budget. Turning off thinking made the problem disappear, and it was concluded that using per-request thinking_budget_tokens is better.

Also, -b 2048 / -ub 2048 gave better prefill than the default for prompts of 2K tokens or more, and in the 8GB environment, the configuration of offloading more MoE layers to CPU via --n-cpu-moe 99 is the currently chosen setup. However, the optimal n-cpu-moe split is still an open question.