Browse / Sensitive Data Disclosure / AIID-1020
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts
- Class
- Sensitive Data Disclosure (filed by a model that read the source)
- Technique
- Real-world incidents
- Affected
- Meta, Users OF Llama.cpp Servers (incident)
What happened, in short
Misconfigured public llama.cpp servers exposed users' prompts, including roleplay scenarios describing sexual abuse of children, to anyone who found them. (Our summary, written by a model. The source text is below.)
Details
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
References
Source: AI Incident Database, used under CC BY-SA 4.0.