Automation of AI Research

The author estimates the probability of no-human-involved AI R&D emerging by the end of 2028 at over 60%. The assessment is that AI is moving beyond merely assisting with code and into a stage where it designs, implements, and evaluates the next generation of models on its own.

While a finished version is not expected within 2026, the author believes that within 1-2 years, a proof-of-concept could emerge in which a model trains its successor at a lower level on its own. The critical threshold is the moment AI research begins to run in a closed loop without human involvement, after which prediction becomes nearly impossible.

At the core of the evidence are code capability and autonomous task duration. SWE-Bench has risen from around 2% at its early launch to 93.9% recently, and METR's time horizon has grown from 30 seconds (GPT-3.5) to 4 minutes (GPT-4), 40 minutes (o1), 6 hours (GPT 5.2 High), and 12 hours (Opus 4.6).

Research reproduction and experiment automation have also progressed rapidly.

• CORE-Bench jumped from 21.5% to 95.5%.
• MLE-Bench rose from 16.9% to 64.4%.
• On PostTrainBench, AI produced performance gains roughly half the level of humans.
• On LLM training optimization tasks, speedups grew from 2.9x to 52x.

AI is also increasingly taking on tasks such as kernel optimization, PyTorch-to-CUDA conversion, Triton kernel generation, and writing kernels for non-standard hardware. Lastly, Automated Alignment Research has shown that AI agents can produce results that exceed human-designed baselines, though this remains close to a small-scale proof-of-concept for now.

While it's acknowledged that individual benchmarks carry noise and bias, combining multiple metrics shows that most of the processes needed to produce today's AI systems already fall within the scope of automation. This leads to the conclusion that end-to-end automation of AI R&D is closer than expected.