AI Briefing#mamba
The latest AI and developer news about #mamba, with the original source and a short summary.
The latest AI and developer news about #mamba, with the original source and a short summary.
A new recipe distills Pythia-1B into a Mamba model, achieving a perplexity of 14.01 compared to the teacher's 13.86.
The model achieves ~1.8% average WER on Open TTS Leaderboard English sets and supports 51 style tags plus zero-shot cloning.
Improves SSM kernel performance by up to 1.8x at batch size 256 and speculative decoding throughput by up to 2.1x at batch size 512
Combining DEER with generalized teacher forcing enables stable, parallel training on time series exceeding 1 million steps, significantly outperforming Mamba.
The Amazon Bio Discovery team released three AI-based antibody research papers covering binding strength prediction, developability assessment, and novel drug design.
New research demonstrates that combining diverse mechanisms (especially including SSMs) has a greater impact on performance than the placement order of attention mechanisms in Transformer stacks.
NVIDIA has released Nemotron-3 Puzzle-75B, an MoE-based model that dramatically improves inference efficiency.
vLLM has resolved the unimplemented TurboQuant error affecting Qwen 3.5+ models caused by an issue with Mamba layers.
AI21 Labs' Jamba model addresses the long-context processing limitations of Transformers through a hybrid architecture.
AI21 hosted a hackathon leveraging Jamba's 256K context window, showcasing innovative AI use cases.
AI21 has unveiled the **Jamba 1.5** family, a high-performance open model lineup based on an SSM-Transformer architecture.
A 32-bit overflow in vLLM's Mamba CUDA kernel caused logprob mismatches.
AI21 reduced padding overhead by about 90% without changing the model.
The cause of rare gibberish was traced to vLLM's Mamba scheduler and cache.
AI21 has open-sourced Jamba2, splitting it into two models: 3B and Mini.
Falcon-H1-Arabic, an Arabic-specialized model with a hybrid architecture combining Mamba and Transformer, has been released.
By leveraging high-quality reasoning data, a 15B reasoning model was distilled into a 2.1x faster Mamba hybrid without performance degradation.
FuriosaAI presented 6 research papers at ICML and ACL that enhance model efficiency, reasoning capability, and flexibility.
Since Mamba, hybrid LLMs have been replacing the limits of Transformers.
AI21 presented grounding as the key challenge for enterprise AI.
Based on 3 years of field experience, AI21 lays out GenAI's limitations and Jamba's solution.
AI21 Labs has released a Jamba-based long-context course with DeepLearning.AI.
TII has released the Falcon 3 open model family, featuring under-10B-parameter models with enhanced math, coding, and reasoning capabilities.
TII has released a 7B-scale Mamba-based model that achieves high performance without an attention mechanism.