AI Briefing#llama-cpp
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
Kolibri-1 is a 78B-parameter Mixture-of-Experts model with 3.46B active parameters, optimized for German and English reasoning.
The new STQ1_0 format achieves 1.3125 bits per weight, reducing the model's storage footprint from roughly 1.5TB while maintaining performance comparable to IQ1_M.
The new GDN chunked kernel implementation in llama.cpp delivers performance gains ranging from 0.9% to 20.6% across various Qwen models and hardware configurations.
The update introduces a playground tab for testing decision models like Kev and OpenJev with probability bars.
The experimental C-based Janas-LLM achieves 24 tokens/s generation for the 48.4GB Qwen3-Next-80B-A3B on a 32GB RAM laptop, significantly outperforming llama.cpp.
The new superchip laptops feature up to 128GB unified memory and 1 petaflop of AI compute, launching with optimized apps from Adobe, GitHub, and major game titles.
The new Jeff-Code agent uses a small decision model to pre-fetch information and manage thinking levels, reducing average task time to 0.68x of the baseline with no significant drop in accuracy.
The open-source engine claims up to 2x faster performance than llama.cpp by compiling kernels specifically for the user's hardware.
Victoria achieves 70.0% on Terminal-Bench 2.1 using REAP pruning, while Maple improves Canadian regulatory accuracy from 6.6% to 21.8%.
Hugging Face Transformers has released a feature that allows direct inference of GGUF models in a compressed state without dequantization on Apple Silicon Macs, utilizing llama.cpp's ggml Metal kernels.
Hugging Face has added native support for the GGUF format to the Transformers library, enabling debugging and flexible use of locally quantized models.
The open-source dynamic quantization tool 'Dynamic Quantiser' has been released, offering higher accuracy than standard quantization in VRAM-constrained environments for llama.cpp-compatible GGUF files.
Hugging Face Transformers now directly supports the GGUF format from llama.cpp, simplifying local inference on Apple Silicon.
This analysis compares the features and suitable workloads of major open-source inference stacks such as llama.cpp, vLLM, and SGLang.
NVIDIA expanded its local AI ecosystem at IFA 2026 by unveiling local inference optimization tools and RTX Spark PCs.
llama.cpp now supports running the Nemotron-3-Puzzle-75B-A9B model, which adopts NVIDIA's hybrid MoE architecture.
Vellium v1.1.0 released an update supporting local STT/TTS and automatic llama.cpp detection.
A new llama.cpp branch featuring Metal and n-gram SSD offload optimizations has been released, improving Qwen-3.8-Flash-Next inference performance on Apple Silicon.
The addition of ConvRot quantization to llama-cpp-turboquant brings the quality of Q6 quantized models close to Q8 levels.
SyzygyResearch has released the 7GB Mach-1-Additive-35B model and a dedicated llama.cpp fork.
The DFlash developer released DFlash 2, which supports Qwen 3.8 27B and Muse Glimmer, and submitted a llama.cpp pull request.
This summary covers the latest updates on low-bit quantized models, including 1-bit, 2-bit, and Ternary models, and the status of optimized inference engines.
llama.cpp has released version v0.1.0, adopting semantic versioning instead of its previous build number scheme.
Support for the new inference model Ling 3.0 has been merged into llama.cpp.