Percepta introduces Spotlight architecture to separate intelligence from memory

Percepta has introduced Spotlight, a new architecture that separates the intelligence module from memory. This design allows knowledge and skills to grow without changing the model's weights, enabling the model to gain new capabilities without retraining.

Key Technical Features

• Unbounded Memory with Constant Access Cost: Spotlight replaces standard attention with a memory system where every token reads from and writes to an unbounded memory. Because the model learns to index individual memory cells, each token only touches a small number of them, keeping access costs constant regardless of memory size.
• Arbitrary Sparsity: Unlike Mixture-of-Experts models that activate a fixed number of experts from a fixed set, Spotlight is arbitrarily sparse. The fraction of memory used can shrink as needed, independent of how the memory grows.
• Writable and Self-Managed Memory: The memory holds knowledge, procedures, and working state. The model itself decides what to load and when to overwrite it, token by token.

Implications

The intelligence module remains the same size while memory expands. This separation means the model's capabilities are not limited by the size of its intelligence module, allowing for dynamic skill acquisition and knowledge retention without the computational overhead of traditional weight updates.