llama.cpp Adds MiMo V2.5 Support
Key point
llama.cpp has submitted a PR to support the MiMo V2.5 model.
Details
PR #22493 in ggml-org/llama.cpp adds text-to-text inference support for MiMo V2.5(+ Pro). The audio and vision configurations for the non-Pro model are excluded from this scope.
- The author has not yet re-verified the
Promodel, but expects conversion and quantization to work correctly. convert_hf_to_gguf.pywas fixed to correctly dequantize FP8 safetensors.- MiMo's TP-aware sharding and fused
attention_qkvwere split apart again to align with the existing MiMo V2 Flash path. attention_value_scalefromconfig.jsonwas also wired into the GGUF conversion path to reduce discrepancies with the transformers implementation.
In Q8_0 verification, Mean PPL(Q) 5.135221 was nearly identical to the base 5.128919, and Mean KLD 0.012455 showed small quantization error.
In the comments, a conversion error occurred due to a missing GGUFWriter.add_attn_value_scale, but the author added a fix commit to address it.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.