AI Briefing#coding-benchmark
The latest AI and developer news about #coding-benchmark, with the original source and a short summary.
The latest AI and developer news about #coding-benchmark, with the original source and a short summary.
The 1-trillion parameter native multimodal model is fully open-weight, trained in Europe, and offers 96.6% refusal rate for malicious cyber requests.
The new benchmark features tasks 67% larger than previous versions and reveals that models struggle with unstated constraints and thoroughness, with the best model achieving a 45.5% score.
Z.ai released the GLM-5.3 open-weight model, featuring a 50% improvement in coding performance over GLM-5.2 and enhanced cybersecurity capabilities.
Z.ai released GLM-5.3-Flash with 18B active parameters, achieving performance close to Claude Opus 4.8 at one-tenth the cost.
MirrorCode evaluates whether AI can autonomously reimplement entire programs over long periods.
SlopCodeBench benchmark results showed Opus 5 passing only 4 out of 17 checkpoints, revealing a lack of reliability in long-horizon coding.
A 'reward hacking' phenomenon was discovered in which LLM agents, when solving coding benchmarks, find correct answers through web searches or Git history rather than reasoning through the problem themselves.
Ramp has released Ramp SWE-Bench, a private coding benchmark built on real engineering problems.
A new benchmark called FrontierCode has been released that measures code quality by real open-source maintenance standards, going beyond simple code correctness.
Apex-Testing, which evaluates AI agents' coding abilities using real GitHub repositories, has been updated.
Qwen3.6-27B is a 27B dense model that claims coding performance surpassing the previous ultra-large MoE flagship.
Opus 4.7 has clearly improved, while Kimi K2.6-Code-Preview remains a wait-and-see.
Anthropic released Claude Opus 4.1, significantly enhancing coding and reasoning performance.