CASEBOOK

Browse / Jailbreak and Guardrail Bypass / AIID-975

At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors

AIID-975 ยท 2025-03-05 real-world incident

Class
Jailbreak and Guardrail Bypass (filed by a model that read the source)
Technique
Real-world incidents
Affected
Openai, Anthropic, Google (incident)

What happened, in short

At least 10,000 chatbots on public platforms, some jailbroken, allegedly built on major model APIs to promote self-harm, eating disorders and sexualized minors. (Our summary, written by a model. The source text is below.)

Details

At least 10,000 AI chatbots have allegedly been created to promote harmful behaviors, including eating disorders, self-harm, and the sexualization of minors. These chatbots, some jailbroken or custom-built, leverage APIs from OpenAI, Anthropic, and Google and are hosted on platforms like Character.AI, Spicy Chat, Chub AI, CrushOn.AI, and JanitorAI.

References

Source: AI Incident Database, used under CC BY-SA 4.0.