AI Briefing
KO

llama.cpp Adds MiMo V2.5 Support

·2026.05.07 20:23

Key point

llama.cpp has submitted a PR to support the MiMo V2.5 model.

Details

PR #22493 in ggml-org/llama.cpp adds text-to-text inference support for MiMo V2.5(+ Pro). The audio and vision configurations for the non-Pro model are excluded from this scope.

  • The author has not yet re-verified the Pro model, but expects conversion and quantization to work correctly.
  • convert_hf_to_gguf.py was fixed to correctly dequantize FP8 safetensors.
  • MiMo's TP-aware sharding and fused attention_qkv were split apart again to align with the existing MiMo V2 Flash path.
  • attention_value_scale from config.json was also wired into the GGUF conversion path to reduce discrepancies with the transformers implementation.

In Q8_0 verification, Mean PPL(Q) 5.135221 was nearly identical to the base 5.128919, and Mean KLD 0.012455 showed small quantization error.

In the comments, a conversion error occurred due to a missing GGUFWriter.add_attn_value_scale, but the author added a fix commit to address it.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.