Alibaba OpenGCUA, Python Harness Boosts OSWorld-v2 Average Score by 19.6 Points

·2026.10.28 19:00

Key point

The OSWorld-v2 average score rose from 45.3 to 64.9 points through harness changes alone, without model training.

1 / 8

Details

  • OpenGCUA, released by the Alibaba MAI-UI team, is a general-purpose computer use agent harness that controls GUIs, shells, and APIs using Python programs instead of clicks.
  • Without model training, harness changes alone raised the average score of 8 models from 45.3 to 64.9 points, a 19.6-point increase, across 82 OSWorld-v2 tasks.
  • The average number of steps per task decreased by 53.0%, and GLM-5.3-Flash showed the largest improvement, rising from 29.6 to 64.9 points.
  • A real-device harness that runs on macOS 14 or later and a Cross-Device SDK are provided.

Technical Features and Results

  • It adopts a 'harness-model decision pre-assignment' structure where the model determines the observation target, action, and task organization.
  • Executing Python code, including loops and conditionals, improved efficiency compared to existing batch methods.
  • GPT-6 Astra recorded a score of 75.7, but the cost per task was approximately $15.26, which was higher than Qwen3.8-Flash, which recorded 60.5 points (approximately $0.38).

Limitations and Cautions

  • Confirmation gating for critical actions has not yet been implemented, so use is recommended only in state-loss-tolerant environments.
  • Analysis suggests that the pure contribution of code execution may be limited, as scores rise significantly with shell access alone.
  • The published OSWorld examples are reference code only, not a full evaluation runner, and using computer agents on machines with business data is inappropriate.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.