MemT-1.0 Introduces Memory-Native Foundation Models with Gated DeltaNet-2 Architecture

Details

MemT-1.0 is a new memory foundation model architecture that internalizes memory as part of model computation, built on the Metis foundation.
At a 128K token history length, MemT-1.0-Nano delivers a 1.86x end-to-end inference speedup and a 66x reduction in persistent storage compared to full context KV cache.
The model family includes MemT-1.0-Nano and MemT-1.0-Flash, trained on 34B tokens using a two-stage pipeline of continued pre-training and post-training.
Evaluation results show the models outperform parametric memory baselines on average and remain competitive with retrieval-augmented methods while largely preserving general capabilities.
The system employs Gated DeltaNet-2 for stable token-level memory updates and Mixture-of-Storage to scale native memory capacity.
By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, offering a practical step toward memory-native evolution.
Resources are available on arXiv, GitHub, and Hugging Face under the MemTensor organization.

Key point