16GB M3 Qwen35B 8.9TPS

Downloaded Qwen3.5-35B-A3B-UD-IQ3_XXS.gguf from unsloth/Qwen3.5-35B-A3B-GGUF and ran it with llama-server on a 15-inch 16GB M3 MacBook Air.

The key was --mmap. This option was described as the core of getting it running.

The main run settings were as follows.

• --ctx-size 4096
• --n-gpu-layers 0
• --parallel 1
• --flash-attn on
• --threads 6
• --batch-size 512
• --ubatch-size 128
• --cache-type-k q4_0
• --cache-type-v q4_0
• --no-warmup

The measured throughput was 8.9 TPS. It was noted that cache-type-k/v q4_0 isn't required but was chosen to reduce memory usage.

After launching, it was noted that the http://127.0.0.1:8081/v1/ API and the http://127.0.0.1:8081 llama.cpp WebUI become available.