People Overestimate AI Confidence

Researchers from the University of Waterloo and UCL, in a study published in Communications Psychology, compared the confidence people attribute to humans versus AI across 7 pre-registered experiments (valid N=52–113).

• Participants watched a human or an AI perform a visual perception task and rated how confident each agent appeared to be.
• Accuracy, task difficulty, and response speed all shifted confidence judgments, but even when showing identical behavior, AI appeared more confident than humans.
• This bias was replicated in visual perception and general knowledge tasks, and weakened in tasks with greater subjectivity, such as emotion classification.
• Additional manipulations revealed that this illusion stems from prior beliefs that AI will be more competent, rather than from actual performance.

The researchers noted that LLM-based conversational agents like ChatGPT and Gemini can lead users to overtrust their recommendations if confidence is not explicitly conveyed, and emphasized the need to design transparent confidence signals.