The Anti-Singularity

Starting from the premise that the next paradigm could be LLMs endlessly iterating on complex designs to produce heuristics, this piece raises the question that the future might not be the singularity we expect, but an anti-singularity. The core idea is not a world where a single general artificial intelligence (GAI) solves every problem, but a world scattered with task-specific heuristic optimizers.

The traditional singularity follows a utopian narrative running from GAI → recursive self-improvement (RSI) → superintelligence (SAI). In the anti-singularity, there is no deeper unified theory of intelligence, and as with Darwin's blind evolution, purposeless search and chance produce the outcomes.

Biology is a prime example of this view. Even with the vast dataset that is nature and enormous capital, drug development tends to end in losses of billions of dollars, and we still haven't managed to digitally reproduce even the simplest life forms. While AI startups receive valuations in the trillions of dollars, biology startups remain marginal by comparison.

Discrete mathematics shows a similar pattern. Simple rules like Rule 30 give rise to complex patterns, and Wolfram's A New Kind of Science expected that nature and science could be reduced to computation, but in reality, because of computational irreducibility, there are almost no efficient general solutions. In the end, trial and error—actually running things to see what happens—becomes the best approach.

In this world, AI remains powerful. If the best method itself is testing many possibilities, then AI, which can attempt vastly more than humans, holds an overwhelming advantage. However, the problem of AI alignment shifts from controlling a single SAI to managing an ecosystem of heuristic optimizers each tuned to different environments. Because these heuristic optimizers are adapted to local environments, they may be less dangerous than a single superintelligence, but because of computational irreducibility, it becomes harder to explain why something went wrong.

• The unit of concern shifts from a universe-wide catastrophe to a localized failure where a particular agent, oddly, churns out too many paperclips.
• Humans are no longer beings who rest after the last invention, but gardeners who tend to and fix a diversity of agents.
• If the anti-singularity is what actually happens, the intuitions and heuristics humans have accumulated over 3.5 billion years could become more valuable, making diversity, resilience, and adaptability more important than preparation-style thinking premised on money becoming meaningless after the singularity.

The order of likelihood is presented as good singularity > anti-singularity > bad singularity. However, which one arrives cannot be changed, and the piece leaves as open questions what indicators would distinguish the two futures, what responses would work for both, and how this relates to the current turmoil in the field of mathematics.