Google Cloud AI Research and Others Unveil Harness²: 11.5-Point Improvement in Agent Performance Without Ground-Truth Labels

Details

Google Cloud AI Research and Northwestern University unveiled Harness², a framework for recursively improving agent harnesses without ground-truth labels
On WorkBuddy-Bench, the score for the GLM-5/OpenCode combination increased by 11.5 points, from 64.2 with the base harness to 75.7
It adopts modifications by comparing only behavioral contrast and accumulates experience in domain-specific libraries to pass on to the next task
With k=5 parallel recursion, WorkBuddy-Bench reached 80.8 points and JobBench saw improvements of up to 7.5 points, outperforming existing methods

Technical Architecture and Experimental Results

Proposed a harness consisting of 8 files, including system prompts, skills, and scripts, and improved it through trial runs and combination stages
Benchmark scores increased for all 5 models, even when the same model was used for both the executor and the editor
Token usage per JobBench task increased from approximately 5M with the base harness to over 100M at k=5

The open-source code assumes a Linux, Python 3.12, and Google Cloud Vertex AI environment
Behavioral comparison and model judgment do not guarantee correctness, nor is it guaranteed that modifications will transfer to other tasks
When running Codex, the sandbox is disabled and there are no network access restrictions, requiring specific host blocking before evaluation