OpenBMB MiniCPM-V 4.7 Achieves 20.51 tok/s on iPhone 18 Pro Max via Lookahead Expert Scheduling

·2026.10.21 22:12

Key point

The system uses Lookahead Expert Scheduling to preload weights, achieving 6.07x speedup over the baseline with only a 0.16-point benchmark drop.

OpenBMB's MiniCPM-V 4.7 runs a 35B-A3B MoE model on iPhone 18 Pro Max at 20.51 tok/s, a 6.07x improvement over the 3.38 tok/s baseline.
The Lookahead Expert Scheduling (LES) technique predicts next-layer expert needs and preloads weights from flash storage during current computations, avoiding expert skipping or routing changes.
Optimized weight layouts and mixed-precision quantization reduce expert weights by 26.7% with a 0.16-point drop across five benchmarks.
The implementation, including LES and DSpark speculative decoding integration, is open source in llama.cpp-omni and achieves around 3 GB peak memory in low-memory configurations.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.