billion-context: Processing 18.7B tokens with a 100K context

A proxy that sits between coding agents and model APIs to hierarchically compress conversation history. The model directly determines the timing and scope of compression, converting it into high-fidelity summaries instead of hard truncation. This allows a single session to be maintained over several days even within a 100K context window.

Unlike existing host-embedded summarizers, the compression is incremental and reversible. Summarized ranges can be immediately restored when needed, and the prefix cache hit rate is maintained at 95–97%. The cost increase due to compression itself is limited to less than 2%, reducing token costs by up to 5x.

It supports various clients such as Claude, Codex, Gemini, and Zed, integrating via native plugins or certificate MITM. In a production environment validated by over 174,000 model calls over 4.5 months, there were zero window violations for 204,800-token models.

It resolves the issue of session interruptions caused by context overflow in long coding sessions. It scores 16.6% higher on the LongMemEval-S benchmark than RAG methods, and independent session costs are reduced by 71.1% compared to the full context. It reduces the burden of context management for long-term projects or work on large codebases.