Skip to content
Bramble

Technology

OpenAI adds CSAM image classifier and 24-hour ChatGPT suspensions for abuse-seeking prompts

OpenAI announced an image classifier to detect child sexual abuse material, including AI-generated images, plus ChatGPT warnings and 24-hour suspensions for users who try to access such content, and new tools for developers and law enforcement.

2 outlets · 0L · 1C · 1R First reported Account updated
Image: TechCrunch
Image: Fox News

1 / 2

The story, neutrally told

Centre · 1OpenAI announced a set of tools to combat child sexual abuse material (CSAM): an image classifier, warnings and temporary blocks in ChatGPT, guidelines for developers, and tools for law enforcement. Mixed · 2The classifier can detect CSAM images, including AI-generated ones, by looking for visual signs of abuse in images not seen before and assigning a score to guide human reviewers. Mixed · 2OpenAI says it developed the classifier with the National Center for Missing and Exploited Children (NCMEC) and is offering it at no cost, through its API, to select or vetted organisations; it is working with Tech Coalition and ROOST to share access.

Mixed · 2In early testing with partners, OpenAI says the classifier flagged 48% fewer images while finding the same amount of abusive material, and flagged images were about twice as likely to be abusive. Right · 1Fox News adds results from outside organisations: AI platform Fal said reviewers confirmed 72% of high-confidence flags as previously unidentified CSAM, and SafetyKit said the tool detected 90% of cases involving drawings or animations that sexualize children. Fox notes these figures come from OpenAI and participating organisations, and that independent testing will be needed. Centre · 1In ChatGPT, users attempting to access material that sexualizes children will see a warning, be directed to specialised support to stop harmful behaviour, and have their account suspended for 24 hours; repeated offences could lead to permanent removal. ChatGPT already blocks CSAM content and related requests.

Right · 1Fox News adds that OpenAI will also remove the flagged conversation, that repeated violations may bring longer restrictions or reports to authorities, and that the warnings target attempts to find abusive material, while existing policy already bans accounts and reports attempts to generate or upload CSAM to NCMEC. Centre · 1OpenAI says it shaped the warning messages with organisations and researchers including Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan of the University of Tasmania, who independently evaluated a trial of deterrent messages. Mixed · 2OpenAI is publishing guidelines for developers to prevent, detect and report CSAM, and says early testing has helped a number of developers adopt child safety tooling and register to report to NCMEC. It is also giving child-safety law enforcement AI-powered tools, training and a field guide; Fox adds that OpenAI has joined the Tech Coalition's Lantern programme and that a first tool helps authorised teams review lengthy chat logs obtained through legal processes.

Mixed · 2TechCrunch places the announcement against lawsuits blaming OpenAI's technology for harming users, especially children, Common Sense Media's 'unacceptable risk' label for teens, and misuse of Grok to create sexualized imagery of children. Fox cites NCMEC data that its CyberTipline received more than 400,000 reports involving generative AI in 2025, though not all involved creating or sharing AI-generated abuse material. Right · 1Whether the warnings actually deter future abuse remains to be shown; Fox says this will require continued evaluation.

Every sentence links to the reporting it rests on. The pill in front of each says where its sources sit: Left, Centre or Right when one side supplies at least half of them, Mixed when they are evenly split. The number is how many outlets it cites.

Left0 outlets

No left outlet in our sources has covered this story yet.

Centre1 outlet

Framing
A concise news report of the announcement, setting it against lawsuits over harm to children and the misuse of other chatbots such as Grok.
Emphasis
The list of new tools, the 48% test result, partners and the surrounding legal and safety criticism of OpenAI.
Leaves out or plays down
Does not give outside partners' results, NCMEC's report figures, or the caveat that results are self-reported.
For example
“OpenAI has had several lawsuits against the company blaming its technology for harming users, especially children.” — TechCrunch

Right1 outlet

Framing
A Fox News CyberGuy consumer-advice column opening with a hypothetical teen deepfake scenario and addressing parents, ending in an endorsement of OpenAI's steps.
Emphasis
How the tool works, parent takeaways, partner results (Fal, SafetyKit), NCMEC's 400,000 reports figure and the caveat that results are self-reported.
Leaves out or plays down
Does not mention the lawsuits against OpenAI, Common Sense Media's rating or the Grok imagery cases.
Charged language
“The scale of the problem is alarming”
For example
“OpenAI is taking steps in the right direction, and I hope other AI companies follow its lead.” — Fox News
“The scale of the problem is alarming.” — Fox News

Ask about this story

Loading…