The Best AI Coding Agent Fully Finishes Only 26% of Real Jobs

 

Your AI coding agent just demoed a full app built from one prompt. Impressive — until you hand it a real codebase and a real change request. A new benchmark built from actual user sessions shows the gap between the demo and reality is enormous: the best AI coding agent on the market fully finishes only about one task in four.

What Co-Design Bench Actually Tested

Researchers from Alibaba, Fudan University, and several other Asian universities  built Co-Design Bench from the messy reality of coding work, not from tidy test suites. They started with 37,463 real user requests drawn from 26,314 coding sessions and distilled them into 273 tasks. The agents had to edit existing web apps — changing layouts, fixing interactions, updating content — sometimes working from screenshots, audio notes, videos, or documents, the way real developers actually hand off work. Ten leading models were tested, including OpenAI's GPT-6 Astra, Claude Opus 5, and Qwen 3.8-Max.

The Best Model Still Leaves 3 in 4 Jobs Unfinished

GPT-6 Astra came out on top, meeting 69.9% of each task's requirements on average. But "mostly done" is not done: it produced a fully correct result in only 26.4% of tasks. On smaller multimodal edits it finished 41%; on whole-repository changes, under 18%. And when a single request mixed three or four kinds of changes — layout plus design plus interaction plus content — performance collapsed below 18% across the board. In nearly every case the agent did the bulk of the work and left at least one detail unfinished, which means a human still has to clean up before anything ships.

Why the Old Benchmarks Were Lying

The striking part is the gap. On older tests like WebCompass-edit, the same models score  above 99% — a difference of more than 35 points. Those benchmarks test polished, single-purpose tasks. Co-Design Bench tests the compound, ambiguous requests real users actually make.  Independent evaluator METR found the same pattern earlier: real-world acceptance of AI-generated  code runs about 24% below SWE-bench scores. The demo numbers were never measuring real work.

What This Means for You

None of this means coding agents are useless — doing 70% of a task in seconds is still a huge productivity win. But plan for the last 30%: budget human review time, keep the agent on well-scoped jobs, and don't let a green checkmark on a benchmark convince you the code is shippable.  One caveat in the paper's favor: every task was a single fixed request, with no follow-up questions allowed, so agents working interactively with a developer may do better than 26.4%.  The honest takeaway stands either way — the last mile of coding still belongs to humans.