AI Briefing
KOSign in

Alibaba Co-Design Bench: Top coding agent GPT-6 Astra completes only 26.4% of real user requests

·2026.11.09 10:42

Key point

The benchmark evaluates ten models on 273 tasks derived from actual user interactions.

Details

  • A new coding-agent benchmark, Co-Design Bench, was released by researchers from Alibaba Group Holding and several Asian universities on arXiv on October 26.
  • The benchmark consists of 273 tasks constructed from real user requests.
  • GPT-6 Astra achieved the highest performance among the ten models tested, fully completing 26.4% of the tasks.
  • The study highlights the gap between current coding agent capabilities and the complexity of real-world user demands.
benchmarkalibaballm-evaluationarxivgpt-6-astracoding-agents
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditSWE-Pair Benchmark: DeepSeek V4.1 Flash Outperforms GPT-6 Luna on Dual-Bug Fixing TasksRedditFrontier3DBench evaluates 11 coding agent configurations for 3D world generationOpenAI BlogBioBasicsBench: GPT-6 Astra scores 66.7% on wet-lab tasks, outperforming GPT-5.6 SolPyTorchKRMIT CSAIL Uses GPT-6 Astra to Directly Edit Weights, Achieving 48.1% on Qwen3.5-0.8B Math BenchmarkHacker NewsOpenAI Disbands Robotics Team, Pivots to General Models for Embodied AIHuggingFace BlogAi2 releases XYBench, a benchmark for testing LLMs' ability to identify and correct user misconceptions