AI Briefing#sparse-attention
The latest AI and developer news about #sparse-attention, with the original source and a short summary.
The latest AI and developer news about #sparse-attention, with the original source and a short summary.
The method maintains near-full-KV accuracy while supporting 700B scale models at 96-99% of DSA throughput.
Spotlight enables infinitely growing memory without increasing access costs by using an arbitrarily sparse indexing mechanism.
The 'Declarative Attention' technique has been released, allowing language models to declare areas requiring attention during their reasoning process, reducing computational costs by up to 52%.
The GLM-5.3 model has been visualized on HF Viewer, confirming that while its architecture remains identical to GLM-5.2, performance has improved through training enhancements.
The SGLang Diffusion team accelerated the MiniMax-H3 video generation model by up to 6.24x on eight NVIDIA H200 GPUs using Cache-DiT and SubBlock sparse attention.
LongCat-Flash-Lite-Sparse weights have been released, supporting up to 1M tokens of context.
CalibAtt is a technique that speeds up video generation models by up to 1.58x by identifying unnecessary connections during attention computation.
The base model of DWARF, a new Sparse Attention architecture aiming for O(1) complexity, has been released.
The paper proposes SBM-Transformer, which adjusts attention sparsity based on data to improve computational efficiency.
An open-source kernel has been released that enables efficient training of MiniMax Sparse Attention (MSA) on Hopper and Blackwell GPUs.
MiniMax M3 uses Sparse Attention technology to dramatically reduce cost and computation even with long contexts, enabling the implementation of long-horizon agents.
Keye-VL-2.0, a 30B-scale multimodal model specialized for long-video understanding and agentic capabilities, has been released.
MiniMax has released MSA, a Sparse Attention technology that supports processing long contexts of over a million tokens.
MiniMax has announced MSA, a new sparse attention technology that maximizes computational efficiency for large-scale context processing.
MiniMax has launched the M3 model featuring a 1 million-token context and multimodal capabilities, with weights to be released within 10 days.
China's MiniMax has released M3, a multimodal model that captures both performance and cost-efficiency through its innovative MSA architecture.
MiniMax has unveiled **MiniMax M3**, an open-weight model that combines coding, 1M context, and native multimodal capabilities.
MiniMax has previewed its M3 model, which introduces a new sparse attention mechanism to achieve 15.6x faster long-context response speed.
RTPurbo, a technique that converts a Full Attention LLM into a highly efficient Sparse model with only a few hundred short training steps, has been unveiled.
Core-periphery block-sparse attention cuts the high-resolution cost of ViT.
Selective hierarchical attention was about 17x faster than standard attention at 512K context.
An analysis of DeepSeek-V4's architectural innovations, infrastructure optimizations, and the trial-and-error process during training.
Qwen2.5-Turbo has expanded its context length to 1 million tokens and significantly improved inference speed and cost efficiency.
DeepSeek-V3.2-Exp adds DSA and cuts API prices by more than 50%.