GitHub Copilot Applies 4 Optimizations to Boost Task Completion Efficiency Over Token Reduction

The efficiency of AI coding agents depends not on the token count of individual interactions, but on how quickly and accurately tasks are completed through the correct context. GitHub Copilot implemented four changes to address the issue where pursuing only token reduction led to retries due to missing information, thereby increasing costs and time.

The Pitfall of Token Reduction

Simply shortening the output length of tool calls can be inefficient. For example, when shell outputs were shortened using utilities like RTK (Rust Token Killer), the model lost necessary information, resulting in cases where it had to re-read the original source or re-execute commands. This resulted in shorter individual responses but increased the total token usage and time required for the overall task.

Noise Compression and Information Preservation Strategy

GitHub Copilot introduced a selective output compressor that reduces repetitive noise while preserving the core information the agent needs to perform tasks. Benchmark tests confirmed that initial excessive compression caused the model to repeat work, leading to the establishment of the following three-tier policy:

• Preserve Original: Outputs from source code or arbitrary commands such as cat and git diff are passed through unchanged.
• Reconstruct Search Results: Search results from tools like grep are grouped into a more efficient format while retaining all items.
• Compress Repetitive Noise: Only predictable, repetitive noise such as installation, build, test, and progress outputs is compressed.

The compressed output provides a path for the agent to immediately restore the original if needed. This restoration path serves as a safety net and a metric to evaluate the appropriateness of the compression strategy by tracking how frequently the agent re-opens the original. These improvements are applied to all products using the same harness, including GitHub Copilot CLI, the Copilot app, and code reviews, to enhance efficiency.