OpenAI introduces MedChemBench to evaluate AI models on medicinal chemistry reasoning
·2026.11.20 09:00
Key point
The benchmark tests 99 problems from published research, revealing that models excel at structure reconstruction but struggle with compound selection and property prediction.
Details
- OpenAI released MedChemBench, a benchmark of 99 problems drawn from 117 papers and patents to test AI models on medicinal chemistry tasks.
- GPT-6.2 Astra achieved 84.5% of available credit at the xhigh reasoning setting, outperforming GPT-6 Astra (83.3%), GPT-6.1 Sol (82.0%), and GPT-5.6 Sol (58.2%).
- Models show strong performance in reconstructing molecular structures (95.4% credit for Astra) and explaining scientific evidence, but score only about half the available credit for selecting compounds and predicting numerical properties.
- The benchmark evaluates four components: molecular structures, scientific reasoning, ranking and selection, and property predictions, using partial credit based on closeness to measured values.
- Limitations include the potential for models to have prior exposure to the published sources and the fact that each response is a single attempt without tools or feedback, meaning results do not confirm real-world drug discovery capabilities.
Benchmark Structure
- 99 problems sourced from 117 papers and patents, featuring 761 distinct molecular structures.
- Tasks require connecting molecular structures with experimental evidence to explain results, compare compounds, and predict properties.
- Evaluates ability to handle tradeoffs, such as balancing strong activity against a disease-related protein with absorption and metabolic stability.
Model Performance Details
- GPT-6 Astra correctly reconstructed an 18-atom ring in all ten responses at the xhigh setting, while GPT-5.6 Sol and GPT-5.6 Luna failed all ten.
- GPT-6 Astra reached 83.8% at max effort, with higher reasoning settings generally improving scores across the eleven evaluated GPT models.
- Performance varies significantly by task type, with structure reconstruction being the strongest area and property prediction remaining a major challenge.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.