AI BriefingLLMs Declare Their Own Attention Scope
Key point
The 'Declarative Attention' technique has been released, allowing language models to declare areas requiring attention during their reasoning process, reducing computational costs by up to 52%.
Details
To address the inefficiency of scanning the entire KV cache during long-context processing, the Declarative Attention (DA) protocol, where the model declares its own attention scope, has been proposed.
While existing external score-based token selection methods still incur O(N) costs per step, DA guides the model to explicitly specify whether to look at the full context, specific regions, or recent outputs during the Chain of Thought (CoT) process. The inference engine parses this declaration like a tool call to skip unnecessary KV cache reads.
Key Results and Features
- Computational Efficiency: Reduced the total number of attention tokens during decoding by 52.0% and 31.1% on the Gemma-4-31B and Qwen-3.6-27B models, respectively.
- Accuracy Retention: Accuracy degradation on 15 long-context tasks was only 1.27pp and 2.75pp, respectively, with the degradation margin decreasing as model size increases.
- Scalability: Presents a new axis for Sparse Attention that is expected to have greater potential when combined with learning-based methodologies.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
'Cache-to-Cache' Framework Revealed for Direct KV-cache Communication Between LLMs Without TextStanford and NVIDIA Release 'Contrastive Language Model' CLM with 9x Faster Inference