AI Briefing#llama-cpp
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
A bug has been reported where llama.cpp's llama-server fails to pass the reasoning_effort parameter to the chat template.
The llama.cpp server now supports executing tools within sandbox containers using Podman or Docker.
llama.cpp has improved prompt processing speed by up to 31% through vectorization using F16C intrinsics.
Meta has released Muse Glimmer, a 30B multimodal model for local agents, under the Apache 2.0 license.
The llama.cpp mainline now supports voice cloning with Qwen3-TTS 1.7B.
The llama.cpp team has unveiled a Mac app and simplified model serving feature.
llama.cpp has fixed a tool calling issue in DeepSeek v4 Flash 0731.
BeeLlama.cpp, a fork of llama.cpp, has released v0.4.1, which improves KV cache quantization efficiency.
Llama.cpp now supports MCP (Model Context Protocol), enabling the construction of agentic environments based on local models.
CachyLLama, a fork of llama.cpp, has dramatically improved prompt processing speed with SSD-based KV caching.
pi version 0.81.0 has added integrated support for llama.cpp (llama-server router).
llama.cpp is preparing an update that boosts prompt processing speed by 15% on ROCm environments and resolves certain quantization bugs.
BeeLlama v0.4.0, a fork of llama.cpp with improved KV cache quantization performance, has been released.
It's an LLM server that maximizes inference efficiency on low-spec CPU environments through KV cache reuse and n-gram archives.
EAGLE3, an inference acceleration technology, has been merged into llama.cpp, improving inference efficiency.
OSCAR, a technique using spectral covariance-aware rotation to reduce quantization loss in KV Cache, has been released.
A 4B-scale Granite Vision 4.1 model specialized for document structure extraction has been added to llama.cpp.
A PR applying post-norm hidden states to improve Qwen model MTP performance has been submitted to llama.cpp.
An update has been merged into llama.cpp to support the Mellum and Granite embedding models.
mistral.rs v0.8.2 has been released, delivering up to 2.8x faster CUDA inference performance than llama.cpp on the latest GPUs such as H100 and B200.
Support for the EXAONE 4.5 model has been added to the llama.cpp open-source project.
A Qwen3.6 GGUF model has been released that integrates an MTP head, enabling self-speculative decoding without a separate draft model.
The llama.cpp project has released a unified `llama` binary and an official website (llama.app).
A major update for AMD/ROCm environments has been released in llama.cpp version B9387.