Strengthening ChatGPT's Ability to Respond in Sensitive Conversation Situations

OpenAI worked with mental health experts to update the default model in ChatGPT so that it can better recognize when a user is experiencing psychological distress, de-escalate the conversation, and, when appropriate, help them seek professional care.

This safety improvement focuses on three key areas:

• Mental health issues (such as psychosis or mania)
• Self-harm and suicide
• Emotional reliance on AI

For future model releases, in addition to existing safety metrics related to suicide and self-harm, OpenAI plans to add emotional reliance and non-suicidal mental health emergencies to its standard safety test suite. To this end, the Model Spec has been updated to establish guidelines for the model to respect users' real-world relationships, avoid reinforcing unfounded beliefs, and pay attention to indirect signals of psychological distress.

Because mental health-related risk situations occur very rarely, OpenAI runs offline evaluations alongside real-world usage data. This is meant to deliberately set up high-risk scenarios to uncover the model's vulnerabilities and precisely measure how well the model responds to risks that might not otherwise surface in real-world settings.