Mode Collapse: You're Not Exempt Either

mode collapse is a phenomenon where a generative model skews toward the most common outputs in its training distribution. The more you retrain on AI-generated data, the more small biases accumulate across generations, and the distribution increasingly locks in on the most common side.

In an image generator example, a symmetric split like 50:50 allows resources to be divided evenly, but once one side becomes more common, like 70:30, the model chooses the side it can draw better, and that choice returns as an even stronger bias in the next training stage. So in ambiguous situations, the model leans toward dogs, where the gain from getting it right outweighs the loss from missing, and the more it retrains, the more dogs it produces.

• A grant-making organization that receives proposals split 70:30 between global health and animal welfare tilts further toward global health, which is easier to evaluate, reaching 75:25, and newly hired evaluators inherit that same bias.
• A band starts with 7 pop songs and 5 rock songs on its first album, shifts to 9:3 on the next album, drops rock entirely by the third, and only tries a new direction like IDM on its fifth album once it has some breathing room.
• Division of labor isn't fully explained by the gains from trade alone. Like accountants and personal trainers, the skill that gets rewarded repeatedly strengthens faster, while other options fall increasingly behind.

The key is slack. Without time and resources, you can't maintain unfamiliar options or learn new skills, and like hunting versus fishing on a remote island, one side becomes increasingly advantageous, eventually leading to more extreme specialization. This tendency also appears in evolution, and a footnote notes that targets that are denser or simpler and easier to generate are favored more.