AI Briefing#llama-cpp
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
The latest AI and developer news about #llama-cpp, with the original source and a short summary.
On the RTX 5060 Ti 16GB, Qwen3.6-35B-A3B ran about 10% faster than the prebuilt version.
Combining an RX 9070 and RTX 5070 Ti, the author ran Qwen 35B Q4 at over 100tps.
On dual RTX 5060 Ti GPUs, Qwen 3.6-35B-A3B hit 21.7 tok/s at 90K context.
In the Qwen3.6-35B-A3B GGUF benchmark, Unsloth quants claim the best size efficiency relative to KLD.
Qwen3.6-35B-A3B-UD-Q2_K_XL handled 58 tool calls at a 98.3% success rate on a 16GB VRAM laptop.
The uncensored Qwen3.6-35B-A3B Aggressive version and K_P GGUF quants have been released.
Comparing Vulkan and ROCm on a Strix Halo 128GB machine showed the winner varying by model.
A case of running Qwen 3.6 35B on an RTX 2000 Ada 16GB at 400 t/s prompt and 24 t/s generation.
Using turbo3, achieved about 40 tok/s running Qwen3.6 35B A3B at 260k ctx on a 12GB VRAM 3080.
llama.cpp's ggml added graph_reused to reduce CUDA's O(n) change checks.
A local LLM learns your Claude Code approval habits and auto-allows tool calls.
On an RTX 5080·64GB Windows 11 setup, Llama.cpp was about 2x faster than LM Studio.
A tuning log running Qwen3.5-35B at 64k context on an RTX 4060 Ti 16GB, hitting 40-60 tok/s.
It compiles English function descriptions into 22MB neural programs that run locally on llama.cpp.
A dynamic expert cache in llama.cpp boosted Qwen3.5-122B-A10B token generation speed by 27%.
Gemma 4's tool calling and dialog compliance have improved, requiring a Jinja template update.
When an agent read papers and forks before writing code, llama.cpp optimizations improved significantly.
A hands-on report and workarounds for Gemma 4 being weak at system prompts and tool calling.
NVIDIA has optimized Gemma 4 for local execution on RTX, DGX Spark, and Jetson.