Mac Mini M4 Pro 70 t/s
Key point
On a Mac Mini M4 Pro 64GB, the 14B dense model hit 16.2 t/s, while Qwen3.6 35B NVFP4 reached 69.8 t/s.
Details
Running qwen3.6:35b-a3b-coding-nvfp4 on a Mac Mini M4 Pro 64GB with the combination of Ollama 0.22.1 and MLX native NVFP4 produced an average of 69.8 t/s. This version mattered because the MLX runner for NVFP4 was broken in 0.21.x.
A comparison using the same prompt and 120-token output is as follows.
Qwen2.5-14BGGUF Q8_0 + llama.cpp: 16.2 t/sQwen3.6-35BGGUF Q5_K_M + llama.cpp: thinking ON 27.3 t/s, OFF 27.5 t/sQwen3.6-35BNVFP4 + MLX native: 69.8 t/s
The difference between 32k and 8k context was under 0.1 t/s, and the difference between q8_0 and f16 KV cache was also around +0.3 t/s. For short tasks under 500 tokens, shrinking the context window had almost no effect, and since the MoE model's actual active parameters were only about 3.5B, it ran faster than the dense 14B model.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.