Up to 25% Additional Savings Over Existing KV Compression Methods, With Improved Performance — CASK

In LLMs' long-form reasoning, KV cache grows rapidly with decode length, straining memory and inference stability.

Existing KV compression for reasoning has mainly relied on making token importance scoring more sophisticated to discard low-ranked entries, but CASK argues that improving the scorer alone does not substantially reconstruct the actual keep-set.

Instead, it reframes the problem as behavior-preserving structured consolidation.

• Splits the decode-time reasoning trace into a protected core and a mergeable scratch
• Preserves the core as the region that underpins answer formation and intermediate states
• Applies selective consolidation only to scratch to reduce redundancy
• In prompt-heavy settings, decode compression may start late, so it uses a two-stage structure of prefix eviction + decode-stage consolidation

In evaluation, on the H100 reasoning gate, it showed higher full-KV continuation fidelity than TriAttention at the same budget, and on AIME24 and AIME25, the crossover cask@384 > triattention@512 appeared repeatedly.

In prompt-heavy replay, multi_news and vcsum were used as decode-active witnesses, while qmsum and gov_report revealed the prefix budget exhausted boundary.

The key conclusion is that performance in reasoning KV compression comes more from the combination of core preservation + scratch consolidation than from more complex scorer engineering.