CASEBOOK

Browse / Jailbreak and Guardrail Bypass / AIID-661

Leonardo AI's Platform Alleged to Have Been Used for Creating Nonconsensual Celebrity Deepfakes

AIID-661 ยท 2024-03-26 real-world incident

Class
Jailbreak and Guardrail Bypass (filed by a model that read the source)
Technique
Real-world incidents
Affected
Leonardo AI (incident)

What happened, in short

Users shared prompts that bypassed Leonardo AI's content moderation, using its text-to-image generator to create nonconsensual sexual images of celebrities. (Our summary, written by a model. The source text is below.)

Details

Sydney-based startup Leonardo AI's text-to-image generator was alleged to have been exploited to create nonconsensual sexual images of celebrities, bypassing content moderation systems with user-shared prompts.

References

Source: AI Incident Database, used under CC BY-SA 4.0.