Automated Alignment Researchers: Scaling Scalable Oversight with LLMs

As the pace of AI model advancement accelerates, the problem of Scalable Oversight—how to supervise models smarter than humans—has emerged as a core challenge. In particular, once models begin generating code that is too complex for humans to understand, it becomes very difficult to judge whether the model is behaving as intended.

To address this, Anthropic researched a Weak-to-strong supervision approach. This involves using a relatively weak model as a 'teacher' to fine-tune a more powerful 'base' model so it can achieve optimal performance. The goal of the research is to measure how much Performance Gap Recovered (PGR) the strong model achieves through feedback from the weak model.

The research team built Automated Alignment Researchers (AARs) to test whether Claude could conduct alignment research on its own. They created an autonomous research environment by equipping 9 Claude Opus 4.6 models with the following tools:

• Sandbox: a space where the model can think and work
• Shared Forum: a space for sharing research results with other models and collaborating
• Storage: a system for uploading written code
• Remote Server: a server that provides a PGR score for each idea

Based on different initial guidance, the AARs autonomously propose their own ideas, design experiments, and analyze results, exploring ways to accelerate the pace of alignment research on their own.