The Oracle and The Firm

For the latest AI models to perform complex tasks, the ability to handle vast numbers of tokens — Context Management — is essential. Currently, OpenAI and Anthropic are taking opposite approaches to solving this problem.

OpenAI uses a Compaction approach. The Codex model leverages server-side compaction technology that removes or summarizes unnecessary information from long conversation flows, leaving only the core information. This maintains high consistency within a single thread and plays a role like an 'Oracle' that helps users remember details.

Anthropic, on the other hand, prefers a Splitting approach. The Claude model has a structure where a problem is divided into multiple sub-problems, and each Sub-agent performs its task within its own context window, passing only the results to the parent agent. This is similar to the operating method of a 'Firm,' where each team member within an organization carries out their assigned work and reports back.

This difference in strategy shows distinct characteristics in terms of efficiency.

• OpenAI: High consistency centered on a single thread and efficient use of K/V caching.
• Anthropic: Increased cost due to the possibility of duplicate work occurring between agents, and a high perceived speed.