Introducing LifeSciBench

Existing life sciences AI evaluations have focused heavily on simple fact recall or standardized problem-solving, limiting their ability to reflect the complexity of actual research settings. This is because researchers must interpret incomplete evidence, reconcile conflicting results, design experiments, and make decisions under uncertainty.

LifeSciBench was designed to bridge this gap, and includes 750 tasks written and reviewed directly by 173 Ph.D.-level experts with drug development experience in the bio and pharmaceutical fields. Beyond simple knowledge-based answers, this benchmark measures 7 workflows: evidence processing, analysis, design and optimization, scientific reasoning, verification and operations, translation, and scientific communication.

The dataset's key characteristics are as follows:

• Complex reasoning: 79% of all tasks require multi-step reasoning or decision-making processes averaging 4 steps.
• Diverse data sources: The benchmark includes 1,062 Artifacts (charts, PDFs, tables, sequence files, chemical structure files, web references, etc.), with 53% of tasks requiring interpretation or synthesis of these materials.
• Rigorous validation: All tasks underwent expert review, securing scientific grounding based on strong expert consensus with over 90% agreement among reviewers.

Evaluation is conducted through detailed Rubrics (evaluation criteria). Based on a total of 19,020 detailed criteria, the benchmark precisely measures whether models can perform scientific claims, calculations, decisions, and justifications at an expert level.