AI Briefing#benchmarks
The latest AI and developer news about #benchmarks, with the original source and a short summary.
The latest AI and developer news about #benchmarks, with the original source and a short summary.
Unsloth reports its Dynamic 3.1 GGUFs for DeepSeek-V4.1 outperform GSQ-RCO by 10-20% on agentic tasks at 512K context, while statistical analysis suggests GSQ-RCO's claimed 99.80% recovery for IQ2_XXS is nearly impossible.
The model achieves 6 confirmed strengths across 11 categories, including a #1 rank in Humanity's Sixth Sense and #2 in SWE-Bench Pro V2.
The model ranks #1 on the Artificial Analysis Intelligence Index v5.0 at 59 and leads in vulnerability detection with 88.2% on SEC-bench Pro.
AI21 analysis of internal traffic reveals public coding benchmarks overestimate task difficulty, leading to inefficient model routing.
Nemotron-3-Ultra-CC scored 535.4/600 at IOI 2026, surpassing the top human score, while a specialized IMO system achieved 30/42 points.
The new model costs around 75% less than Haiku 4.5 and introduces adjustable effort settings for the first time in the Haiku class.
Human intelligence emerged only 300,000 years ago, a blink in the 3.7-billion-year history of life, suggesting it is not an evolutionary upper bound.
GPT-6.1 Sol is available now in ChatGPT Work and Codex for Plus, Pro, Business, Enterprise, and Edu users, with API access via gpt-6.1-sol and cached input priced at $0.10 per million tokens.
Opus 5.5 scores 58 on Artificial Analysis Intelligence Index but costs $5.98 per task compared to GPT-6 Astra's $3.26 due to higher token usage.
A roundup of new local and open-source multimodal models including Kimi K2.6 and Qwen3.6.
On M3 Ultra, Qwen 3.6 35B achieved a 100% tool-calling success rate across most of the 5 agent frameworks.
GPT-Rosalind has been released as a reasoning model to help with life sciences research and drug development.
On refined data, 1.7B beat 744B, but noisy traces collapsed performance.
Opus 4.7 improved on coding but regressed significantly on long-context reasoning.
Even the latest AI agents show significantly lower performance when using skills under realistic conditions compared to benchmarks.
Shares how to split up models by role for OpenClaw based on PinchBench scores.
The perplexity NaN cause in some MiniMax M2.7 GGUF was investigated and fixed.
Qwen-Image is a 20B image model strong in complex text rendering and precise editing.