AI Briefing
KOSign in

IBM Research CUGA Outperforms Hermes and Deep Agents on AppWorld Benchmark

·2026.11.04 00:57

Key point

IBM's open-source CUGA agent harness achieved an 81.1% task completion rate on the AppWorld benchmark, significantly outperforming Hermes and Deep Agents using the same GLM-5.3-flash model.

1 / 2

Details

  • IBM Research released CUGA (ConfigUrable Generalist Agent), an open-source enterprise agent harness designed for sovereign cloud and air-gapped deployments, which outperformed competing harnesses in controlled testing.
  • In AppWorld benchmark tests using the GLM-5.3-flash model, CUGA achieved an 81.1% task completion rate (TGC) and 66.7% scenario completion rate (SGC), compared to 46.7% TGC for Hermes and 30.0% TGC for Deep Agents.
  • CUGA implements a policy layer that enforces rules at runtime via four execution points: Intent Guard, Playbook, Tool Approval/ToolGuard, and Output Formatter, moving governance constraints from system prompts to the harness architecture.
  • The harness utilizes task decomposition, tool shortlisting, and CodeAct execution to reduce model decision points and catch errors before final output, with results reproducible via the provided harbor-eval repository.
open-sourceenterprise-aiagent-harnessibm-researchcugaappworld-benchmark
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

PyTorchKRKOMOS Achieves SRCC of 0.900 in Korean Speech MOS PredictionGeekNewsHerdr releases native desktop app for managing terminals and coding agentsRedditQwen3.8-Flash-Next Reaches Closed-Model Robotics Performance via Open-Source uni-robot-agent HarnessHuggingFace BlogIBM Research Palette: 20B open-weight model ranks second on PresentBench, first in Economics and EducationGitHub BlogArnis open-source project converts OpenStreetMap data into Minecraft worldsHacker NewsOctomind: Open-source Rust AI agent harness outperforms Claude Code on benchmarks