AI Briefing#gguf
The latest AI and developer news about #gguf, with the original source and a short summary.
The latest AI and developer news about #gguf, with the original source and a short summary.
On the RTX 5060 Ti 16GB, Qwen3.6-35B-A3B ran about 10% faster than the prebuilt version.
In the Qwen3.6-35B-A3B GGUF benchmark, Unsloth quants claim the best size efficiency relative to KLD.
The uncensored Qwen3.6-35B-A3B Aggressive version and K_P GGUF quants have been released.
Compared MMLU scores of GGUF quantized models in a 24GB VRAM environment.
A tuning log running Qwen3.5-35B at 64k context on an RTX 4060 Ti 16GB, hitting 40-60 tok/s.
The perplexity NaN cause in some MiniMax M2.7 GGUF was investigated and fixed.
A router mode has been added to the llama.cpp server that allows multiple models to be dynamically loaded and switched without restarting.