AI Briefing#kv-cache
The latest AI and developer news about #kv-cache, with the original source and a short summary.
The latest AI and developer news about #kv-cache, with the original source and a short summary.
This article introduces engineering optimization techniques that leverage the Hybrid SWA architecture of the MiMo-V2.5 model to reduce KVCache storage space and compute to about 1/7 the level.
ReFreeKV, a threshold-free KV Cache compression technique that automatically adjusts the compression budget based on input data, is proposed.
Based on HBM bandwidth and pipeline architecture, this forecasts how model parameter sizes will change through 2031.
Proposes a new infrastructure model for sharing and trading pre-computed KV Cache to reduce redundant computation by AI agents.
InfiniteKV has been released, dramatically extending a model's context length by compressing the KV cache and storing it in RAM or on disk.
A lightweight retriever that compresses DeepSeek-V4's CSA KV-cache to maximize GPU memory efficiency has been released.
OSCAR, a technique using spectral covariance-aware rotation to reduce quantization loss in KV Cache, has been released.
An update has been applied to llama.cpp that improves inference performance for Gemma-4 models by preventing KV cache copying.
A study shared Transformer's QKV projections and dramatically reduced the KV cache.
A feature has been proposed for llama.cpp that dynamically changes the KV cache quantization level without reloading the model.
This introduces Speculative KV coding, a technique that uses a predictor model to losslessly compress an LLM's KV cache by up to 4x.
Huawei has unveiled KVarN, a new KV-cache quantization backend that improves both capacity and throughput for vLLM at the same time.
**OCTOPUS**, which maximizes transformer KV Cache compression performance by combining octahedral parameterization with joint quantization, has been released.
Wall Attention has been released, applying per-channel multiplicative decay to support efficient training and inference.
llama.cpp has fixed an issue where quantized KV cache did not work when using --sm tensor in multi-GPU environments.
The MiMo V2.5 series maximizes long-context processing and multimodal performance through Hybrid SWA and MoE.
Shard has been released, compressing the KV cache memory of Llama-3.1-8B by about 10x without performance degradation.
A CUDA-based FWHT has been added to llama.cpp, improving inference speed when using KV-cache quantization.
A 2-bit quantization technique has been released that compresses KV cache memory usage by about 7x while minimizing performance degradation.
A bug in llama.cpp causes the KV cache of the Qwen3.6-35B-A3B model to be dropped, significantly degrading inference speed.
The latest open-weight LLMs have adopted KV cache compression and attention optimization techniques for long-context efficiency
A problem where CUDA acceleration is limited when using asymmetric KV cache quantization in llama.cpp, along with solutions, is being discussed.
DeepSeek is pursuing a grand strategy that goes beyond simply selling models, aiming to build a $10 trillion Chinese AI hardware ecosystem.
vLLM concluded that FP8 KV-cache is the default choice, while TurboQuant's higher-bit options offer little real benefit.