AI Briefing#ssm
The latest AI and developer news about #ssm, with the original source and a short summary.
The latest AI and developer news about #ssm, with the original source and a short summary.
Improves SSM kernel performance by up to 1.8x at batch size 256 and speculative decoding throughput by up to 2.1x at batch size 512
New research demonstrates that combining diverse mechanisms (especially including SSMs) has a greater impact on performance than the placement order of attention mechanisms in Transformer stacks.
To overcome the context-scaling limits of Transformers, a 'sleep' mechanism is proposed that compresses recent information into fast weights.
AI21 Labs' Jamba model addresses the long-context processing limitations of Transformers through a hybrid architecture.
H Company has unveiled Holotron-12B, a computer use agent model that applies a hybrid SSM and attention architecture to maximize inference efficiency.
FuriosaAI presented 6 research papers at ICML and ACL that enhance model efficiency, reasoning capability, and flexibility.
The Falcon-H1 model series, featuring a hybrid architecture combining Transformer and SSM, has been released.
TII has released a 7B-scale Mamba-based model that achieves high performance without an attention mechanism.
AI21 has released Jamba, a Mamba-based model supporting a 256K context window.