AI Briefing2-bit KV Cache Quantization Technique 'OSCAR' Released
Key point
A 2-bit quantization technique has been released that compresses KV cache memory usage by about 7x while minimizing performance degradation.
Details
OSCAR (Offline Spectral Covariance-Aware Rotation), a new technique for 2-bit KV cache quantization, has been released. This technique captures Q/K/V activation values through a small calibration set, estimates attention-based K/V covariance offline, and applies an orthogonal rotation for each layer.
The key features are as follows.
- Memory Efficiency: It can compress KV cache memory footprint by about 7x.
- Performance Retention: Based on the GPQA benchmark, it shows only a minor accuracy drop of single-digit perplexity (pp) in dense reasoning models.
- Ease of Use: By providing pre-computed rotation matrices (
.ptfiles), it can be applied immediately without users needing to perform complex eigendecomposition themselves.
Optimized rotation matrices are currently provided for Qwen3 (4B, 8B, 32B) and GLM-4.7-FP8 models.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
LLMTraceFX Introduces Independent Auditor to Verify That Cache Hits Do Not Prove Computation Omission
KV Cache Research: Limitations of LRU Alternatives and Failure in Capacity-bound EnvironmentsSalesforce Releases 'Random Attention' KV Cache Eviction Policy for Reasoning Models