HELIX Engine open-source agent runtime reduces token consumption by 77.73% in 50-task SWE benchmark

HELIX Engine, an open-source context and memory management system for coding agents, reports a 77.73% reduction in total provider tokens on a 50-task software engineering benchmark. The system reduces redundant inference by retiring historical context and streamlining deterministic tool flows without modifying model weights.

Benchmark Results

The evaluation compared HELIX V3 against a vanilla baseline using Sol 6.1 High on 50 tasks:

• Token Efficiency: Total known tokens dropped from 50,686,068 (vanilla) to 11,288,140 (HELIX), a 4.49× reduction.
• Input Tokens: Reduced by approximately 77.85%.
• Output Tokens: Increased by 37.14%.
• Model Requests: Increased from 341 to 420.
• Correctness: Verified solves decreased from 38/50 to 34/50.

Mechanism and Limitations

HELIX operates by separating model reasoning from execution bookkeeping. Key components include historical context retirement, which removes completed interactions from active replay while keeping originals recoverable, and deterministic tool flow for authorized mechanical operations. The author explicitly notes that the 4-task regression is a significant active research problem. The benchmark used a historical baseline rather than a randomized A/B test, and results may not generalize to other models or workloads. Current research focuses on recovering correctness and reducing model-request amplification.