One Root Cause

It views an LLM as a structure that maintains a consistent reasoning path over given premises, rules, and context, and views LoRA-based continual learning as a structure that rearranges existing and new knowledge, then proposes that the two phenomena may share the same root.

The core idea is that preserving a structure requires satisfying the specific conditions that structure demands. This is formalized as a minimal model of structural persistence and verified through two experiments.

• LLM reasoning degradation: as contradictory information accumulates within a conversation, maintaining logical reasoning became harder.

• However, when contradictions were externally organized and structured, i.e., by separating what was true in the past from what is true now, performance remained more stable.

• This suggests that the accumulation of unresolved contradictions, rather than length itself, better explains the collapse.

• LLM catastrophic forgetting: when learning new information, the model tends to update in a way closer to overwriting rather than gradually accumulating existing knowledge.

• In particular, coherence breaks down significantly when a change to a given premise requires modifying all related knowledge that depended on that premise.

In summary, the argument is that both reasoning performance degradation and catastrophic forgetting can be seen as "collapse that occurs when the conditions for maintaining an existing structure are broken," and that a solution for one could also connect to the other.