#state-space-models

The latest AI and developer news about #state-space-models, with the original source and a short summary.

Apple Research Proposes Two-Stage Distillation to Convert Transformers to Mamba

A new recipe distills Pythia-1B into a Mamba model, achieving a perplexity of 14.01 compared to the teacher's 13.86.

·2026.11.11 22:00

Apple Research Proposes Weighted State Souping for Efficient Long-Context Retrieval

The method learns query-dependent weights to compose cached state encodings, improving performance on document QA, needle-in-a-haystack retrieval, and tool use.

·2026.11.10 09:00

Apple Research Finds Gating Mechanism Causes State Space Models to Prioritize Memorization Over In-Context Learning

The study demonstrates that gating mechanisms in SSMs cause models to learn in-weights memorization solutions, delaying or preventing convergence to correct in-context learning even without architectural memory limits.

·2026.11.10 09:00

AWS introduces priming to convert pre-trained Transformers into hybrid state space models with 99% less pretraining cost

The method reduces pretraining costs by over 99% and enables controlled comparisons of hybrid architectures like Gated KalmaNet, Gated DeltaNet, and Mamba-2.

·1·2026.10.13 02:00

Apple's Machine Learning Research at ICLR 2026

Apple unveiled research on RNNs, SSMs, multimodal, 3D, and protein folding at ICLR 2026.

·2026.04.22 09:00

Infinitely Further: Tool-Use Unlocks Length Generalization for State Space Models

SSMs equipped with tool-use generalize beyond their length limits to problems of arbitrary length.

·2026.03.27 09:00