FST Framework Proposed for Continual Learning

LLM parameter updates (Slow weights) absorb task-specific information, but cause Catastrophic forgetting and reduced plasticity. In contrast, in-context learning (Fast weights) is fast and cheap, but has limits in performance improvement.

This research proposes the Fast-Slow Training (FST) framework, which combines these two approaches.

• Slow weights: Preserve general reasoning ability by maintaining model parameters.
• Fast weights: Rapidly learn task-specific information from text feedback through optimized context.

The key results of FST are as follows:

• Shows up to 3x higher sample efficiency compared to existing RL methods on reasoning tasks, reaching higher performance thresholds.
• Reduces KL divergence from the base model by up to 70%, mitigating catastrophic forgetting.
• Maintains the model's Plasticity even after training, increasing adaptability to new tasks.
• Continuously acquires new tasks without performance degradation even in Continual learning environments where task domains keep changing.