AI Briefing#llm-architecture
The latest AI and developer news about #llm-architecture, with the original source and a short summary.
The latest AI and developer news about #llm-architecture, with the original source and a short summary.
A new recipe distills Pythia-1B into a Mamba model, achieving a perplexity of 14.01 compared to the teacher's 13.86.
Tom Mann explains that research engineering involves building reliable infrastructure for recursive self-improvement agents, with 90% of time spent on code and 10% on ideation.
The 50-page report details architecture, training, and evaluation for Drex models, which achieve 93.4% accuracy on long-context benchmarks.
Qwenputer, an experimental model adding a RISC-V CPU to Qwen3.5-4B, is released, recording 97.5% accuracy on synthetic arithmetic benchmarks
VHA uses linear operations to halve physical query heads, achieving 4x KV cache reduction and +0.70% quality improvement over MLA on a 30B MoE model.
The new operator uses exponential relevance density over a continuous memory curve to achieve exact normalized attention with closed-form gradients.
The Aimee project has released vLLM plugins for Qwen 3.8 and Gemma4 models, enabling native memory access that decouples knowledge from model parameters and eliminates the need for retraining.
The new FactorEngram architecture improves language modeling on 340M and 1B parameter backbones by using basis-level contextual gating for n-gram memory.
Spotlight enables infinitely growing memory without increasing access costs by using an arbitrarily sparse indexing mechanism.
WarpState, an LLM architecture that processes long contexts using a fixed-size dual associative memory independent of token count, has been released.
An 'Infinite-Parameter LLM' architecture has been proposed, which converts real-time interaction data into weights via a hypernetwork and performs online updates using a Bayesian approach.
OpenArch, an open-source project implementing major LLM architectures like Llama and DeepSeek with a focus on readability, has been released.
In pre-training progress from 2019 to 2025, data improvements provided 3.24 times greater compute efficiency gains than model improvements.
ZEP has introduced a step-by-step quiz generation architecture combining LangGraph and Amazon Bedrock AgentCore to overcome the limitations of single-prompt approaches.
Alibaba previews the Qwen4 architecture, emphasizing reduced inference costs.
Thinking Machines released its first self-developed model, Inkling, under the Apache 2.0 license.
Proposes the Mobius-v0 architecture, which improves training efficiency and inference speed by separating knowledge storage and reasoning processes.
The base model of DWARF, a new Sparse Attention architecture aiming for O(1) complexity, has been released.
The Qwen team announced HydraHead, an architecture that combines FA and LA at the attention-head level to improve long-context processing performance.
Olmo Hybrid model demonstrated superiority over Transformer in semantic token prediction.
The open-source model GLM-5.2, featuring a 1M token context and improved coding capabilities, has been released.
Defines the 'Harness,' the seven core architectural components needed to secure the reliability of AI agents.
Parallax attention has been proposed, overcoming the limitations of existing LLA while achieving performance on par with FlashAttention.
The claim is that because a reasoning model's reasoning trace and final answer are generated from the same computation, the reasoning trace cannot guarantee the faithfulness of the answer.