AI BriefingWhy Language Models Hallucinate
Key point
LLM hallucinations occur because current evaluation methods encourage guessing rather than acknowledging uncertainty.
Details
Large language model (LLM) hallucination refers to the phenomenon of generating information that seems plausible but is not factual. According to recent research by OpenAI, these hallucinations occur not only due to model capability issues but also because standard training and evaluation procedures encourage guessing rather than acknowledging uncertainty.
Many current evaluation methods score models based solely on whether they got the answer right (accuracy). This is similar to how, on a multiple-choice test, guessing is more advantageous than leaving an unknown question blank. If a model answers "I don't know," it receives zero points, but if it guesses even with a wrong answer, it can probabilistically earn points.
Comparing gpt-5-thinking-mini and OpenAI o4-mini through the SimpleQA evaluation metric makes the difference clear.
- gpt-5-thinking-mini: Has a high abstention rate (52%), actively acknowledging uncertainty, but its accuracy is 22%.
- OpenAI o4-mini: Has a slightly higher accuracy of 24%, but its error rate (75%) is very high, resulting in frequent hallucinations.
Ultimately, accuracy-focused leaderboards lead developers to make models guess excessively rather than express uncertainty. To address this, an evaluation system is needed that imposes greater penalties for confident errors and awards partial credit for appropriate expressions of uncertainty.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.