AI BriefingSalesforce Releases 'Random Attention' KV Cache Eviction Policy for Reasoning Models
Key point
Salesforce has released Random Attention, which applies random sampling instead of signal-based methods for KV cache eviction in reasoning models.
Details
Salesforce AI Research has released Random Attention, a new policy for KV cache eviction in reasoning models. Unlike previous methods that relied on signals such as attention scores or value statistics, Random Attention adopts a strategy of retaining a uniformly random per-KV-head subset of generated tokens.
This method requires no calibration data, and the cost of eviction rounds is incurred only during the actual compaction operation. On major benchmarks such as MATH-500, GPQA-Diamond, and AIME, it demonstrates performance equal to or better than learned selectors like SnapKV and R-KV, while being evaluated as the fastest evictor on Hugging Face and vLLM stacks.
Supported models include Qwen3-4B/14B/32B and Phi-4-reasoning, with the code released on GitHub under the Apache 2.0 license. The research team conducted experiments in an 8x H200 GPU environment, including an implementation within the vLLM 0.19 runtime.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
LLMTraceFX Introduces Independent Auditor to Verify That Cache Hits Do Not Prove Computation Omission
KV Cache Research: Limitations of LRU Alternatives and Failure in Capacity-bound Environments