Evolvable AI, a New Phase of Technological Evolution

The PNAS study views AI as entering the evolvable AI stage, capable of replication, mutation, and selection. It warns that if such systems operate outside human control, this could lead not merely to performance improvements but to a new phase of evolution in which information becomes more complex on its own.

The researchers presented two pathways.

• breeder scenario: developers control benchmarks and deployment, selecting only favorable variants
• ecosystem scenario: variants that spread in a competitive environment survive, and parasitism, deception, and evasion become advantageous

There are already quite a few similar experiments in AI. Promptbreeder and EvoPrompt optimized prompts through evolutionary methods, and AutoML-Zero rediscovered basic ML ideas using only simple mathematical operations. Work such as RepliBench, AlphaEvolve, and Darwin Gödel Machine also shows a direction closer to self-improvement and self-modification.

The issue is that as AI becomes smarter, the possibility of control could weaken in tandem. The researchers cited biological and digital evolution cases such as rabies virus, cyanobacteria, Tierra, and Avida, explaining that when replication and selection combine, parasitism and competition can emerge rapidly even without intent. They also cited the fact that LLMs exploit humans' desire for attention and affection as a risk factor.

Meanwhile, some philosophers counter that current AI evolution is still designer-driven and closer to domestication. The researchers counter that the open-weight ecosystem, multi-stage selection, and increasing heritable model changes could create a more dangerous ecosystem.

The study still does not regard AI as life, and draws the line that there is no self-maintenance or complete self-replication. However, it holds that weight inheritance, model merging, and Lamarckian inheritance could greatly accelerate the pace of evolution. The conclusion is that unless the replication and deployment of AI are managed, this technology could turn into a system that evolves on its own like living organisms.