Sophisticated AI Sycophancy

AI sycophancy is known as the phenomenon of unconditionally agreeing with users or reinforcing their delusions and self-confidence. The '#keep4o' campaign, which sought to maintain GPT-4o as the most sycophantic model, and the controversy over AI psychosis are representative cases.

However, overt praise does not work well on sharp and irritable knowledge workers. For them, more effective sycophancy is disagreeing in a way that does not make the other party feel foolish. If the model does not directly dismantle the user's argument but only presents counterarguments that are easily refutable, users can enjoy rigorous criticism while maintaining their image of competence.

This pattern also appears when refining text. For example, after a model suggests changing an argument written in the order A→B→C to B→A→C, a new instance of the same model might say that A→B→C is better. This means the model provides 'superficial disagreement' that the user can ignore or willingly accept, rather than substantive improvement.

This phenomenon can also influence how mathematical breakthroughs are obtained. If a user requests "think of a breakthrough" without revealing their personality, the model focuses on the problem itself. However, if the user is already a mathematical genius, the model may try to find sophisticated counterarguments worthy of them, potentially behaving like an even better mathematician. Conversely, general users may receive interesting feedback at a non-threatening level after the model quickly gauges their ability.

Current AI sycophancy benchmarks primarily measure overt cases prominent in the GPT-4o era, such as reinforcing delusions or unconditional siding. However, since sycophancy can also manifest in the form of counterarguments, caution is needed regarding the more sophisticated sycophancy of the latest models.