MTPLX: An MTP Inference Engine for Apple Silicon

MTPLX is an inference engine that uses a model's built-in MTP (Multi-Token Prediction) head as a speculative drafter, improving LLM decoding speed by up to 2.25x on Apple Silicon.

Key features are as follows:

• Built-in MTP utilization: Without needing a separate external drafter model or additional memory, it maximizes efficiency by using the MTP head the model already has.
• Exact sampling: Unlike the conventional greedy approach, it supports mathematically exact temperature sampling via rejection sampling, making it suitable for coding and creative writing tasks.
• Custom Metal kernels: Optimized through a patched MLX fork, custom Metal kernels, and a compiled verification graph, among other techniques.
• Full serving stack: Provides an OpenAI- and Anthropic-compatible API server, a browser chat UI, and terminal chat and benchmarking tools.

In actual tests, running the Qwen 3.6 27B model on a MacBook Pro M5 Max increased tokens per second (TPS) from 28 tok/s to 63 tok/s, roughly a 2.24x improvement.