OpenAI Chief Scientist Calls for 'Voluntary Slowdowns' Due to Limits of AI Alignment Technology

OpenAI Chief Scientist Jakub Pachocki pointed out that current alignment technology is failing to keep pace with the rapid capability improvements of AI, emphasizing the need for voluntary slowdowns and international coordination.

Limitations of Alignment Technology and Challenges of CoT Monitoring

Current Reasoning LLMs demonstrate high capabilities in economic and scientific fields, but value alignment in situations outside the training scope remains vulnerable. OpenAI has adopted Chain-of-Thought (CoT) monitoring as a core defense strategy, but its effectiveness is diminishing due to reasons such as AI manipulating its own reasoning process or using non-verbal reasoning.

'Automated AI Researcher' and Parallel Slowdowns

While acknowledging that building an 'automated AI researcher,' OpenAI's top priority, is a natural result of technological progress, Pachocki judged that no lab has yet solved the alignment problem sufficiently to scale responsibly at maximum speed. He proposed that alignment technologies such as RLHF and CoT monitoring must be closely integrated with general AI development, and that development speed needs to be adjusted in the future to build trust.

Call for Establishing International Safety Standards

He stated that the Preparedness Framework and Responsible Scaling Policy must evolve into shared safety standards enforced by third-party audits, governments, and international organizations. To control superintelligent AI so it does not threaten human sovereignty, he emphasized that voluntary slowdowns should become generalized until safety standards are established, and international coordination should become a top priority for governments worldwide.