OpenAI releases free CSAM classifier with AI-generated content detection

·2026.11.17 02:00

Key point

The new tool detects AI-generated abuse and is available at no cost to vetted organizations via the OpenAI API.

Details

  • OpenAI released a new image classifier to detect child sexual abuse material (CSAM), including AI-generated content, which is the first broadly available tool with a specific signal for AI-generated CSAM.
  • In evaluations of user-uploaded images, the classifier flagged 48% fewer images while detecting roughly the same number of abusive images as existing protections, with flagged images being twice as likely to contain abuse.
  • Early access partners reported high accuracy: fal confirmed 72% of high-confidence flags as novel CSAM, invideo AI found one in four flagged cases led to a ban, and SafetyKit detected 90% of sexualized drawings and animations with false positives below 1% on legal adult content.
  • The classifier is available at no cost to vetted organizations through the OpenAI API, with access coordinated through the Tech Coalition and ROOST.
  • New ChatGPT warnings interrupt attempts to access abusive material, removing the conversation and pausing access for 24 hours while directing users to support resources.
  • OpenAI published practical CSAM guidance for developers and is providing law enforcement units with AI tools to review chat logs, identify connections, and create timelines.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.