AI2 Releases Benchmark Analysis Method

Allen Institute for AI (AI2) has released BenchMIRT, a new methodology for analyzing actual measured capabilities at the individual prompt level of LLM benchmarks. It was developed to address the issue that while existing benchmarks are designed to measure specific abilities (e.g., safety, reasoning), their individual questions may rely on unintended other abilities or contain mixed signals.

Core Technology and Analysis Method

BenchMIRT applies multidimensional IRT (MIRT), an extension of Item Response Theory (IRT) from psychometrics. This allows for performance analysis at the model and question levels, estimating which latent capabilities are most closely associated with answering specific questions correctly. The research team trained on data from over 34,000 questions across 16 benchmarks and 100 LLMs, independently recovering two major dimensions: safety and general reasoning.

Insights on Existing Benchmarks

The analysis revealed that some benchmarks are more strongly associated with different abilities than intended.

• BBQ: Classified as a safety benchmark testing for social bias, but BenchMIRT analysis shows it is more strongly associated with general reasoning capability. This suggests that low scores may be due to question comprehension and reasoning difficulty rather than safety issues.
• WMDP: Tests dangerous dual-use knowledge in fields like biology and chemistry, but scores are more strongly associated with general reasoning. However, since refusing to provide dangerous knowledge is the desired response, there is an inverse correlation where higher general reasoning ability leads to lower scores.
• HarmBench: Confirmed that within a single benchmark, standard questions and context-based questions mix different signals.

By separating the nuanced capability differences obscured by averaging benchmark scores, BenchMIRT is expected to contribute to improving the accuracy and transparency of LLM evaluation.