Teaching Open Models Scientific Research

Loka, AWS, Arcee AI, and Prime Intellect built reinforcement learning environments so open models can use tools to investigate scientific questions, reason from incomplete evidence, and produce verifiable results. One handles tasks that use biomedical tools to find evidence, while the other handles tasks that infer Gene Ontology annotations from protein evidence and return them in strict JSON.

After running 21 controlled post-training experiments, the Run 120 checkpoint excelled at both tasks and was selected as the final configuration.

• Drug Tool evaluation scores rose from 70.8% to 81.2%, improving at every evaluation point.
• BioReason's composite score reached 0.863, which includes GO term F1, GO tree similarity, aspect coverage, and JSON validity.
• Applying a single round of GEPA prompt search before reinforcement learning improved BioReason validation performance by about 84% and Drug Tool by 7.7%. This was a pre-post-training validation result, separate from the final performance.

The Drug Tool environment consists of 800 training prompts and 200 held-out prompts, spanning 17 workflow categories and a total of 5,049 structured tool calls. The model must select and configure arguments for 7 search tools including PubMed, GEO, KEGG, UniProt, and STRING, recover from failed searches, and submit a concise, synthesized result.

The reward design was built to evaluate not just the final answer but also tool selection, arguments, retrieved facts, task completion, efficiency, and conciseness. BioReason measures biological content, output structure, and JSON validity together.

This approach matters not only for scientific accuracy but also operationally. Small models and LoRA adapters can run within an organization's cloud boundary, be pinned to a specific policy version, and be updated to fit the organization's own data and standards. Teams can directly control the data pipeline, reward functions, evaluation methods, serving costs, and incident response.