AI Briefing#gguf
The latest AI and developer news about #gguf, with the original source and a short summary.
The latest AI and developer news about #gguf, with the original source and a short summary.
MTP support in llama.cpp can boost inference speed for the Qwen 3.6 27B model by 2.5x.
Unsloth released a fixed version of the Mistral Medium 3.5 128B GGUF model that had been producing errors in long contexts.
GGUF quantization files and recommended variants for IBM Granite 4.1 30B have been released.
Mistral has released a unified multimodal model with 128B parameters and a 256k context.
IK_LLAMA has added Qwen3.5 MTP support, and MTP-preserved GGUFs are required.
XiaomiMiMo's MiMo-V2.5 has been quantized into text-only GGUF and released.
llama.cpp has merged initial native NVFP4 MMQ support for SM120, and related GGUF conversions have also been released.
Compared the performance of Qwen 3.6 27B's BF16, Q4_K_M, and Q8_0 quantizations.
Luce DFlash boosted Qwen3.6-27B throughput by up to 2x on a single RTX 3090.
Experimental inference support for DeepSeek v4 Flash has been released in llama.cpp.
A 36B MoE model built with Darwin V7 achieved 88.4% on GPQA Diamond.
On an RTX 5070 Ti 16GB and 32GB RAM, Qwen3.6-35B-A3B Q8_0 achieved 44 t/s.
Max-and-Omnis released GGUF files for Nemotron-3-Super-64B-A12B-Math-REAP.
On a combination of m720q i7, 64GB RAM, and T1000 8GB, Qwen 3.6 27B recorded 20 TPS.
Qwen3.6-35B-A3B ran with 64K context on an RTX 5070 12GB.
Qwen3.6-27B is a 27B dense model that claims coding performance surpassing the previous ultra-large MoE flagship.
An uncensored Aggressive version of Qwen3.6-27B and K_P quantizations have been released.
Unsloth has released a GGUF distribution of Kimi K2.6 along with related documentation.
In Terminal-Bench 2.0, MiniMax M2.7 scored 41.3%, lower than M2.5.
A GGUF Q4_X conversion of Kimi-K2.6 has been uploaded to Hugging Face.
On Gemma 4 26B-A4B GGUF, Unsloth quants top the charts by KLD in most cases.
Unsloth's Mistral Small 4 GGUF has been updated along with chat template fixes.