AI Briefing#llama-cpp
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
The latest llama.cpp build significantly improved the RTX6k Qwen3.5 benchmark.
DFlash for Qwen3.5-35B-A3B was confirmed running on an RTX 2080 SUPER 8GB.
IK_LLAMA has added Qwen3.5 MTP support, and MTP-preserved GGUFs are required.
Native NVFP4 in llama.cpp b8967 boosted prefill speed by up to 68%, but generation speed stayed nearly the same.
XiaomiMiMo's MiMo-V2.5 has been quantized into text-only GGUF and released.
A llama.cpp regression increased the Qwen3.6-27B IQ4_XS size from 14.7GB to 15.1GB.
Resolved Gemma 4 E2B multimodal OOM in llama.cpp on iOS.
A llama.cpp benchmark comparing ROCm 6.4.2 and Vulkan on the 6900 XT was published.
Running Qwen 35B-A3B always-on on a 16GB Mac M4 hit an SSD bottleneck before a RAM one.
Applying ngram-mod to Qwen3.6 27B on the R9700 showed speed improvements, though with some variability.
Box, a security-enhanced Android AI app that integrates LiteRT, llama.cpp, and more to operate fully offline, has been released.
Qwen3.6 35B-A3B showed a prompt processing speed of 282 tokens per second on a Radeon 780M iGPU.
On a combination of m720q i7, 64GB RAM, and T1000 8GB, Qwen 3.6 27B recorded 20 TPS.
llama.cpp has added multimodal support for Reka Edge 2603 along with template fixes.
A setup connecting llama.cpp with VSCode to run Gemma-4-26B-A4B-IT-Q8_0 has been shared, along with its performance results.
Qwen3.6-35B-A3B ran with 64K context on an RTX 5070 12GB.
An uncensored Aggressive version of Qwen3.6-27B and K_P quantizations have been released.
A procedure for running Gemma 4 VLA locally on a Jetson Orin Nano Super was shared.
The little-coder and Qwen3.6-35B-A3B combo scored 78.7% on Aider Polyglot.
TurboQuant fixed a speculative decoding bug in its Qwen3.6 fork.
By optimizing the q1_0/q8_0 dot path, up to 67x throughput improvement was reported on x86.
Unsloth's Mistral Small 4 GGUF has been updated along with chat template fixes.
Llama.cpp token speed measurements on the Zenbook A16 with Snapdragon X2 Elite Extreme.
Boosted Qwen3.6-35B-A3B speed by 54% on an RTX 5070 Ti using --n-cpu-moe.