Preventing Online Child Sexual Exploitation and Abuse

OpenAI is focusing its capabilities on preventing, detecting, and blocking child sexual exploitation and abuse (CSAM/CSEM) involving AI models in order to develop safe and beneficial AGI. To this end, OpenAI applies multi-layered safeguards from the pre-deployment stage all the way through to production models.

Users are strictly prohibited from engaging in the following actions, and violations will result in the blocking of service access:

• Generating and distributing CSAM (regardless of whether it is AI-generated)
• Grooming minors and exposing them to inappropriate content
• Body-shaming minors and inciting them into dangerous challenges
• Sexual/violent roleplay involving minors

If a violation is detected, the relevant account is immediately blocked and reported to NCMEC (the National Center for Missing & Exploited Children). If a developer continues to leave problems unaddressed in their own app, the developer's account itself will be blocked, and OpenAI continuously monitors for attempts to circumvent such blocks.

On the technical side, OpenAI detects and removes CSAM/CSEM starting from the training data stage to prevent models from acquiring related capabilities in the first place. In addition, OpenAI uses hash matching technology and content classifiers from Thorn to detect and block both known and newly emerging abusive content in real time.