Continual Learning for AI Agents

Continual learning for AI agents is not simply about updating model weights. Agent systems can learn across three layers: Model, Harness, and Context.

The Model layer is the stage where the model weights themselves are updated, using techniques like SFT or RL (GRPO, etc.). However, a key research challenge here is addressing Catastrophic forgetting, where existing knowledge is lost when learning new data.

The Harness layer refers to the code, tools, and basic instructions that drive the agent. Recently, approaches like Meta-Harness, where the agent analyzes logs of tasks it has performed to optimize the harness code itself, have been gaining attention.

The Context layer includes instructions, skills, and memory that exist outside the harness. This is divided into agent-level learning, where the agent updates itself, and Tenant-level learning, which provides information customized for each user or organization.