How Pi's Compaction Works

LLM-based coding agents face Context Window limitations as conversations grow longer. When all previous messages and tool call results accumulate, the model eventually becomes unable to process the request.

To address this, Pi uses Compaction technology. Instead of simply starting a new conversation and discarding all previous context, it summarizes the core content of the conversation to compress the context, allowing the conversation to continue.

Pi's Compaction Implementation:

• Trigger: Automatically executes just before reaching the context limit, or can be manually executed by the user via the /compact command.
• Recent Context Preservation: Recent messages are kept as-is without summarization. Pi sets a token budget of approximately 20,000 tokens (about 5–20 turns) to ensure the integrity of the recent conversation.
• Dedicated Prompt Usage: During Compaction, a separate system prompt called 'Context Summarization Assistant' is used instead of the general coding assistant prompt.
• Structured Summary: The summarization request is designed to include Goal, Progress, and Key Decisions, enabling efficient context recovery in subsequent conversations.