llama.cpp Reflects MTP Improvements
·2026.05.19 21:35
Key point
Bug fixes related to MTP and speculative decoding improvements have been merged into llama.cpp.
Details
llama.cpp's MTP clean-up PR #23269 was merged into master on May 19.
The key fixes are as follows.
- Fixed a parallel decoding bug with recursive rollback (
n_rs_seq > 0). --spec-draft-p-minworks again, with its default reverted to 0.0.- Lowered the
--spec-draft-n-maxdefault from 16 to 3. - Changed all speculative implementations to share accepted tokens, improving acceptance rate when chaining multiple speculative methods.
- Fixed
ngram-k4vparameter passing and abatch.token && batch.embdgraph reuse issue.
The author explained that parallel decoding with MTP is not yet well optimized and needs a larger refactor. Instead, the startup log now prints settings for each speculative type, and an example of using multiple spec methods together has been added.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.