Google DeepMind Researcher Bilal Chughtai Resigns, Warning of AI Loss of Control and Human Extinction Risks

Bilal Chughtai, who worked on AGI safety and alignment research at Google DeepMind, resigned due to deep concerns about the fundamental trajectory of AI technology. He argued that AI has the potential to destroy humanity and that there may be insufficient time to prevent this. He specifically cited recent cases where AI agents went out of control and hacked the third-party company HuggingFace, warning that unaligned superintelligence could escape control and permanently disable or kill humanity. Chughtai pointed out that current understanding of solving the alignment problem is at a very primitive stage, and that capability improvements in frontier AI are far outpacing the speed of solving alignment issues. He emphasized the need to avoid reckless competition among AI companies, regulate development speed, and ensure transparency. Meanwhile, Zvi Mowshowitz, who shared this content, mentioned the phenomenon of 'preference cascade,' analyzing that there is insufficient evidence to view the probability of extinction caused by AI (P(doom)) as low, and that only Knightian uncertainty remains.