Agents That Build Better Training Data

We propose Autodata, a general methodology in which AI agents act as data scientists to build high-quality training and evaluation data. This methodology includes a Meta-optimization process that trains the agent itself to generate more powerful data.

As a concrete implementation, we use Agentic Self-Instruct, which has demonstrated improved performance over existing synthetic data generation methods across various fields such as computer science research, legal reasoning, and mathematical object reasoning. In particular, we confirmed that the performance gains become even larger when the data scientist agent itself is meta-optimized.

This Agentic data creation approach presents a new path for converting inference compute into improved model training quality, and holds the potential to bring about a fundamental shift in how AI data is built going forward.