AI Briefing#long-context
The latest AI and developer news about #long-context, with the original source and a short summary.
The latest AI and developer news about #long-context, with the original source and a short summary.
HuggingFace has unveiled SmolLM3, which delivers outstanding performance at the 3B scale and supports a 128k long context.
AI21 released Jamba 1.6, an open model supporting 256K context.
Qwen has released the Qwen2.5-1M series, open-source models supporting up to 1 million tokens of context length.
NVIDIA has released KVPress, a KV cache compression toolkit for handling long context in LLMs.
Based on 3 years of field experience, AI21 lays out GenAI's limitations and Jamba's solution.
AI21 Labs has released a Jamba-based long-context course with DeepLearning.AI.
Infini-attention's performance degrades as memory compression repeats, and Ring Attention or YaRN have been confirmed as more effective alternatives for now.
It presents a strategy for building an agent that handles 1M context using an 8k context model, and using this to train long-context models.
This summary outlines four key approaches that reduce Transformer's computational complexity to handle long contexts.
DeepSeek-V3.2-Exp adds DSA and cuts API prices by more than 50%.