AI BriefingoMLX's tg TPS surges with DFlash
Key point
In the latest oMLX commit, DFlash optimization significantly boosted tg TPS for small contexts.
Details
A benchmark showing DFlash actually delivering results has been published on the latest oMLX commit with DFLASH_MAX_CTX=8192 enabled.
- Model: Qwen3.5-35B-A3B-MLX-MXFP4-FP16
- Key change: tg TPS improved by roughly 2x in small context ranges
- Published performance examples:
- pp1024/tg128: 145.3 tok/s tg TPS, 489.8 tok/s throughput, 21.24 GB peak mem
- pp4096/tg128: 149.0 tok/s tg TPS, 523.3 tok/s throughput, 23.49 GB peak mem
- pp8192/tg128: 70.8 tok/s tg TPS, 537.4 tok/s throughput, 21.51 GB peak mem
- pp16384/tg128: 58.9 tok/s tg TPS, 594.0 tok/s throughput, 22.76 GB peak mem
The author stated that they reproduced these results directly on the latest commit, and added that more benchmarks were posted on the GitHub discussion.
Since this is a case where both latency and token generation speed were improved simultaneously in small prompt ranges, it's a fairly direct signal for anyone interested in optimizing LLM serving on Mac.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Underdog Releases 'Husky', a Model-Specific Inference Engine Up to 4.5x Faster than MLX
Qwen Intelligence Launches Three Mobile AI Agents and Open Benchmark Suite