AI Briefing#benchmark
The latest AI and developer news about #benchmark, with the original source and a short summary.
The latest AI and developer news about #benchmark, with the original source and a short summary.
Gemini Robotics-ER 1.6 strengthens spatial reasoning, instrument reading, and safety.
I-DLM simultaneously boosts DLM quality and speed through introspective consistency.
In a benchmark of 153 web tasks, even the best model achieved only 33.3%.
Across a 90-day tracking of 42 recurring tasks, Opus 4.6's hallucination rate surged.
In Stanford's 2026 AI Index, AI agents scored only about half the level of PhD experts.
AWS and JHU have released a large-scale benchmark for evaluating AI antibody design.
PHP 8.6 will introduce static closure inference and a reuse cache.
TranslateGemma-12b ranked #1 overall across 6 LLM translation benchmarks.
Gemma 4 was connected to the local Codex CLI to verify its tool-calling performance and stability.
An LLM security benchmark for finding known vulnerabilities in real codebases
An open-source memory system with 29 Python scripts and a zero-dependency design that survives compaction was released.
FlashLM v8.3 (6.5M CORTEX) outperformed the v5.2 Transformer baseline under the same 2-hour CPU and same data conditions.
Structural loopholes were revealed in 8 AI agent benchmarks.
GLM-5.1 drew attention with a SWE-Bench Pro score of 58.4 and pricing of $1.4/$4.4.
Claude Opus 4.6 has launched with enhanced coding, long-context, and safety capabilities.
Verified Qwen3.5-122B running at 198 tok/s on 2x RTX PRO 6000 Blackwell.
Relvy has AI execute runbooks to automate on-call response.
The Claw-Eval benchmark, which verifies LLM agents using 139 real-world tasks, has been released.
OpenAI pointed out contamination and test flaws in SWE-bench Verified and recommended discontinuing it.
The Vision2Web benchmark has been released to evaluate the web development capabilities of multimodal coding agents in stages.
Holo3 achieved computer use SOTA with 78.85% on OSWorld-Verified.
It explains LLM performance with 18 ability scores and predicts new task success rates with 88% accuracy.
AsgardBench tests embodied AI's ability to revise plans based on image observations.