AI BriefingBioBasicsBench: GPT-6 Astra scores 66.7% on wet-lab tasks, outperforming GPT-5.6 Sol
·2026.11.06 09:00
Key point
Developed with Ligo, the 100-task benchmark tests AI agents on interpreting raw experimental data, where GPT-6 Astra significantly outperforms previous models.
Details
- BioBasicsBench is a new benchmark of 100 routine wet-lab tasks developed with Ligo to measure how AI agents interpret raw experimental data, identify errors, and recognize when evidence is insufficient.
- GPT-6 Astra achieved a mean reward of 66.7% at the 'xhigh' reasoning setting, compared to 43.8% for GPT-5.6 Sol, indicating progress in routine lab analysis despite remaining gaps in image interpretation and control checking.
- The benchmark covers five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning, using raw inputs like gel images, microscopy stacks, and instrument exports.
- GPT-6 Astra demonstrates improved multimodal handling and context retention, such as recognizing z-stacks and rejecting mislabeled controls, but still struggles with over-interpreting data, discounting contradictory observations, and accurately counting colonies.
- Evaluation involved seven models across 37 reasoning-effort settings, totaling 18,500 results, with tasks created by scientists and reviewed by internal and external experts.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Cat Wu joins OpenAI to lead product management for Core Products