AI BriefingMacBook M5 Rankings
·2026.04.21 06:01
Key point
Compared 21 local LLMs on quality and speed using 164 coding problems.
Details
On a MacBook Air M5, pass@1 was measured using 164 coding problems and an expanded test set to compare code quality, speed (tok/s), and VRAM usage of 21 local LLMs under the same conditions.
- Qwen 3.6 35B-A3B (MoE) ranked #1 with 89.6%, and also posted good speed for a large model at 16.9 tok/s.
- The results showed that a MoE's number of active parameters affects speed, while its total parameter count affects quality.
- By cost-performance/memory efficiency, Qwen 2.5 Coder 7B stood out. With 84.2%, 11.3 tok/s, and 4.5GB, it was concluded to be suitable for everyday coding assistance in 8GB-class environments.
- Gemma 4 underperformed unexpectedly. Gemma 4 31B scored only 31.1%, lower than Llama 3.2 1B (32.9%), and showed a large gap from Gemma 3 27B (78.7%).
- Gemma 4 MoE (26B-A4B) also scored low at 12.2%, and the author noted that this result was confirmed by repeating the measurement multiple times.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Step 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis PerformanceQwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics