AI BriefingMIT Presents Technique to Reduce LLM Overconfidence by 90%
Key point
This introduces the causes of LLMs speaking with confident tones even when hallucinating, and MIT's research findings to address this.
Details
LLMs (ChatGPT, Claude, Gemini, etc.) speaking with confident tones even while causing hallucinations is due to the models' Reinforcement Learning methods.
Standard reinforcement learning only rewards whether an answer is correct, without distinguishing between answers reached through logical reasoning and answers reached through mere guessing. In this process, models learn that answering confidently, rather than revealing uncertainty, earns higher rewards.
This characteristic has also caused problems in actual news reporting. Ars Technica had a case where it included a false quote generated by ChatGPT in an article and later retracted it.
Recently, MIT researchers announced a method to solve this. They demonstrated that by penalizing the gap between the model's claimed Confidence and its actual Accuracy during training, overconfidence can be reduced by up to 90% while maintaining the model's accuracy.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.