AI Briefing
KO

llama.cpp b9180 Adds MTP Support

·2026.05.17 02:01

Key point

The llama.cpp b9180 release adds MTP support and partial rollback.

Details

The llama.cpp b9180 release is out, including Multi-Token Prediction (MTP) support.

Centered on spec: support MTP, it includes cleanup of draft-mtp naming, batch size fixes, and conversion/documentation/format improvements.

GDN partial rollback has been added to the speculative decoding path, allowing rollback up to draft_max even when some draft tokens are rejected.

Server and memory handling were also improved.

  • RS-based MTP is disabled when used together with other spec types
  • Checkpoint logic adjustments and early-exit fixes
  • Added tests for dirty ctx loading

Release artifacts are provided for macOS, Linux, Android, Windows, and openEuler.

releasespeculative-decodingllama-cppllm-inferencemulti-token-prediction
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditComparison of Features and Use Cases for 8 LLM Inference StacksGeekNews3.85x Inference Speedup via Diffusion Draft Generation and AR Verification Based on GLM-OCRTLDR AIThe Efficient Frontier of LLM Inference (6-minute read)TLDR AIvLLM v0.28.0 Released with Major Optimizations for Kimi-K3 and DeepSeek V4Redditllama-cpp-turboquant Adds ConvRot Quantization SupportHuggingFace BlogLFM2.5 Inference Accelerated 3.2x with DSpark