AI BriefingMSA Unveils 100M-Token Memory Model
Key point
MSA has unveiled memory sparse attention that scales up to 100M tokens.
Details
MSA is a latent-memory framework that ties retrieval and generation into a single end-to-end loop. To reduce the bottleneck of existing full attention, it proposes combining document-wise RoPE with sparse attention to treat long document collections like memory.
The core components are as follows.
- Scalable sparse attention + document-wise RoPE keep training/inference complexity nearly linear.
- KV cache compression and Memory Parallel target 100M token inference on 2xA800 GPU.
- Memory Interleave strengthens multi-hop reasoning across scattered memory segments.
In evaluation, the paper tested MS MARCO v1, NQ, DuReader, TriviaQA, NarrativeQA, PopQA, 2WikiMultiHopQA, HotpotQA, MuSiQue, and NIAH(RULER) with the Qwen3-4B-Instruct-2507 backbone. The paper reports that it outperforms the same-backbone RAG, top-tier RAG stacks, and major long-context models, and presents performance degradation of under 9% across the 16K→100M tokens range.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Marker-Inc-Korea Releases 'AutoRAG 2.0', a Local Data Unified Search Agent
Hierarchical Memory Architecture Needed to Resolve AI Inference Bottlenecks