AI Briefing#frontiercode
The latest AI and developer news about #frontiercode, with the original source and a short summary.
The latest AI and developer news about #frontiercode, with the original source and a short summary.
The new benchmark features tasks 67% larger than previous versions and reveals that models struggle with unstated constraints and thoroughness, with the best model achieving a 45.5% score.
Devin Fusion now leads FrontierCode 1.1 Extended with a score of 68.8 at an average cost of $0.60 per task.
A new benchmark called FrontierCode has been released that measures code quality by real open-source maintenance standards, going beyond simple code correctness.