llama.cpp Adds NVFP4/MTP Support

·2026.05.24 03:39

Key point

llama.cpp now supports NVIDIA FP4 quantization and Multi-Token Prediction (MTP) features.

The latest llama.cpp update adds NVFP4 (NVIDIA Floating Point 4-bit) quantization and MTP (Multi-Token Prediction) features.

NVFP4 enables efficient 4-bit inference on NVIDIA hardware, while MTP is a technique that optimizes the model's inference performance.

These features are included in the latest release of llama.cpp (b9297).

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.