AI Briefing
KO

llama.cpp adds MTP support

·2026.05.16 21:11

Key point

llama.cpp added MTP support, and Qwen3.6 MTP GGUF models were released.

Details

PR #22673 was merged into ggml-org:master on 2026-05-16, adding MTP (Multi Token Prediction) head support to llama.cpp.

  • The acceptance rate with 3 draft tokens was about 72%, and on DGX Spark it rose from a baseline of 7 tok/s to 16~21 tok/s.
  • The implementation was verified on Qwen3.6-27B and Qwen3.6-35B-A3B, and it was explained that it could also be applied to other MTP models.
  • Vision input and tensor/pipeline parallelism are compatible, but prompt processing may be slower due to D2H embedding transfer costs.
  • The ggml-org/Qwen3.6-27B-MTP-GGUF and ggml-org/Qwen3.6-35B-A3B-MTP-GGUF model cards specify this PR as a dependency and provide llama.cpp execution examples.
ggufspeculative-decodingllama-cppqwen3mtpmulti-token-prediction
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

Redditllama.cpp Changes Default Loading of MTP TensorsHuggingFace BlogHugging Face Adds GGUF Model Support to TransformersRedditHugging Face Introduces Native GGUF Support in TransformersHacker NewsUnsloth Dynamic 3.0 GGUFsHuggingFace BlogLiquid AI Releases LFM2.5-VL-DSpark: An Experimental Draft Model for Faster Vision-Language InferencePyTorchKRRecipe Released for Serving 320B GLM-5.3-Flash on Free Kaggle TPUs