AI Briefing
KO

llama.cpp Fixes VRAM Leak in MTP Models

·2026.05.22 07:43

Key point

The VRAM leak issue that occurred when using MTP models in the latest llama.cpp build has been resolved.

Details

In b9274, the latest build of llama.cpp, the VRAM leak issue occurring when using MTP (Multi-Token Prediction) models has been fixed.

The existing destroy() function in server_context_impl only cleaned up the main model and context, and had a flaw where it failed to properly release resources such as the speculative decoder (spec), draft context (ctx_dft), and draft model (model_dft).

As a result, each time the server repeated its sleep and resume cycle, new resources kept being allocated without the existing resources being released, which ultimately led to rising VRAM usage and server crashes due to OOM (Out-of-Memory) errors.

With this update, destroy() has been modified to explicitly initialize the related resources in the proper order when called, ensuring stable memory management.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.