AI Briefing#long-context
The latest AI and developer news about #long-context, with the original source and a short summary.
The latest AI and developer news about #long-context, with the original source and a short summary.
A ByteDance Seed paper reveals that DeepSeek V4's chunked KV-cache compression creates periodic retrieval failures, with base model accuracy varying by 40.2 percentage points depending on token position.
The method learns query-dependent weights to compose cached state encodings, improving performance on document QA, needle-in-a-haystack retrieval, and tool use.
The 560B-parameter model supports text, image, and video inputs and achieves 92.21 on OmniDocBench 1.5.
The model resolves slow-channel glare and write sensitivity bottlenecks to train 2.7x faster than Splash Attention at 32K context.
The system achieves 84.33% accuracy on FactKG using 108x fewer tokens than full-context inference while outperforming baselines on five of eight benchmarks.
The disparity in generation cost between later and earlier tokens on frontier open-weight models can reach 7x.
AI2 introduces StateBench, a four-level benchmark revealing that while recurrent architectures generalize better than transformers, top models like Gemini 3.8 Flash and Claude Opus 5.5 still fail on hard state-tracking tasks.
The model supports reasoning, tool calling, and image input, with the first Hy vision capability for reading complex charts and computer use in CLI and GUI environments.
The model uses a looped decoder to execute 56 layer passes from 32 distinct layers, supporting text and image inputs for agentic workflows.
The method maintains near-full-KV accuracy while supporting 700B scale models at 96-99% of DSA throughput.
The model features a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token and a 128K context window.
The model is designed for high-volume agentic tasks, achieving a 76.5% pass rate on SWE-Bench Pro v2-Hard in its initial 0.1 release.
The model supports configurable reasoning, structured outputs, and tool calling, with a fast mode option for latency-sensitive work.
The 3T-class open-weight model supports a 1M token context window and reduces claim-level hallucination rates from 23.4% to 8.3%.
The pipeline reduces average input tokens from 105,183 to 4,886 while maintaining 96.0% accuracy on LongMemEval, outperforming full-history baselines.
VHA uses linear operations to halve physical query heads, achieving 4x KV cache reduction and +0.70% quality improvement over MLA on a 30B MoE model.
The model features a hybrid-linear MoE architecture supporting 256K token contexts and is available on Hugging Face and ModelScope.
The 180B parameter hybrid attention MoE model features a 1,048,576-token context window and selectable reasoning effort levels.
An English-only model supporting 32K context and licensed under Apache 2.0 for commercial use.
The new version adds image input support and an experimental long-context mode for processing documents and logs.
Aleph Alpha has released Kolibri, an open-weight English-German Mixture-of-Experts model with 78.1 billion parameters and support for 1 million token contexts, available under the Apache 2.0 license.
AlephAlpha has released Kolibri, an open-weight Mixture-of-Experts model with 78.1B total parameters and 3.46B active parameters, designed for sovereign and regulated domains with a focus on German language performance.
CoWA and MALA achieve up to 8.6x attention-operator speedups at 128K tokens while maintaining comparable model capabilities to FullAttn.
WarpState, an LLM architecture that processes long contexts using a fixed-size dual associative memory independent of token count, has been released.