δ-Mem: Efficient Online Memory for Large Language Models

δ-Mem is designed to continuously accumulate and reuse past information by adding a small online associative memory on top of a frozen full-attention backbone.

Past information is compressed into a fixed-size state matrix updated via a delta-rule, and its read results are reflected as a low-rank correction to the attention computation during generation. This structure augments memory capability without extending the full context or retraining the backbone from scratch.

In experiments, with only an 8×8 online memory state, average performance improved

• 1.10x over the frozen backbone
• 1.15x over the strongest non-δ-Mem memory baseline

The improvement was larger on memory-focused benchmarks.

• MemoryAgentBench: 1.31x
• LoCoMo: 1.20x

General capability was largely preserved, demonstrating that long-term memory can be efficiently implemented simply by directly connecting a small online state to attention.