AI Briefing#benchmark
The latest AI and developer news about #benchmark, with the original source and a short summary.
The latest AI and developer news about #benchmark, with the original source and a short summary.
This article looks at the latest research trends on how LLMs can reflect and measure diverse cultural values and social biases.
LG AI Research proposed a new benchmark that goes beyond the limitations of existing Gaussian-data-centric approaches to verify mutual information (MI) estimation performance on unstructured data.
LG AI Research proposed a new benchmark that can rigorously verify mutual information (MI) estimation performance in unstructured data settings.
LG AI Research developed BiGGen Bench, a new benchmark for precisely diagnosing the capabilities of language models, and won the NAACL 2025 Best Paper Award for it.
OpenAI's GPT-5.6 Sol achieved first place on the Design Arena benchmark for the first time, showing a new approach that deliberately avoids AI design anti-patterns.
LLMs can significantly improve their code generation ability using only their own outputs, without needing a verifier or separate training.
Papers with Code has launched a dedicated page that consolidates key benchmarks and papers in the robotics and VLA fields.
Hume AI has launched Real World VoiceEQ, a new benchmark that evaluates emotional understanding and naturalness in voice AI.
ALem-World, a new benchmark evaluating LLM agents' long-term collaboration ability, has been released.
The LHTB benchmark has been released to measure whether LLM agents can perform complex tasks spanning hundreds of steps in containerized terminal environments.
Apple's new SpeechAnalyzer API overwhelmed on-device speech recognition performance, recording higher accuracy and faster speed than Whisper models.
OpenAI announced that about 30% of the SWE-bench Pro benchmark contains errors due to design flaws.
The PACE framework accurately predicts performance on costly LLM agent benchmarks using only a small amount of data.
A new benchmark has been released to evaluate AI agents' practical capabilities at a senior engineer level.
Instead of developing a new agent, merging the outputs of existing agents achieved the #1 spot on the Deep Research benchmark.
GLM-5.2 comes close to frontier models in benchmark performance, pushing the boundaries of open models further.
RewardHackBench has been released, using sandbox environments to prevent benchmark cheating by AI agents.
KOLongDoc, a VLM benchmark for evaluating comprehension of long Korean public institution documents, has been released.
A new benchmark called CVE-Bench has been released to evaluate LLM agents' ability to patch security vulnerabilities.
The latest LisanBench update analyzed the performance and efficiency of major Chinese-origin models, including Step 3.5 and Kimi-K2.5.
An analysis suggesting that MLE-Bench performance gains stem more from improved model performance and increased search volume than from algorithmic advances has been released, alongside a new benchmark called FML-Bench.
A study has found that when LLM agents fix security bugs, the most dangerous outcome is a 'plausible but wrong patch' that passes tests while leaving the vulnerability in place.
PolyRange, a new cybersecurity AI evaluation methodology that solves the data contamination problem through an LLM-generation approach, has been released.
PolyRange, an AI security evaluation benchmark that addresses the data contamination problem and reflects dynamic defense environments, has been released.