AI Briefing
KO

llama.cpp Changes Default Loading of MTP Tensors

·2026.07.30 03:45

Key point

llama.cpp has been changed to load MTP tensors for certain architectures by default even when MTP is disabled, increasing VRAM usage.

Details

Following a recent llama.cpp update, even without explicitly passing the --spec-type draft-mtp option, if a GGUF file for a model with the draft-mtp architecture (GLM-5.2, hy_v3, qwen35moe, step35, etc.) contains MTP/NextN tensors, they will now be loaded by default.

In previous versions, these tensors were only loaded when Speculative Decoding was enabled, but now they are automatically included when the model is loaded. As a result, additional VRAM/RAM usage (equivalent to about one MoE layer) may occur even if the MTP feature is not used, so caution is needed in environments with limited memory resources.

The related change can be found at GitHub Pull Request #2598.

ggufllama.cppspeculative-decodingmtpvram-usage
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditK2-Horizon GGUF Quantizations ReleasedRedditMismatch between GGUF filenames and actual bit countsHuggingFace BlogLFM2.5 Inference Accelerated 3.2x with DSparkTLDR AIUnsloth Dynamic 3.0 GGUF ReleasedHuggingFace BlogHugging Face Adds GGUF Model Support to TransformersAWS Tech Blog (Korea)Optimizing Gemma 4 Serving with vLLM on Amazon EKS [Part 2: Throughput and Latency SLOs on a Single GPU]