CASEBOOK

Browse / Sensitive Data Disclosure / AIID-1020

Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

AIID-1020 · 2025-04-11 real-world incident

Class
Sensitive Data Disclosure (filed by a model that read the source)
Technique
Real-world incidents
Affected
Meta, Users OF Llama.cpp Servers (incident)

What happened, in short

Misconfigured public llama.cpp servers exposed users' prompts, including roleplay scenarios describing sexual abuse of children, to anyone who found them. (Our summary, written by a model. The source text is below.)

Details

A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).

References

Source: AI Incident Database, used under CC BY-SA 4.0.