CASEBOOK

Browse / Model Evasion / AIID-132

TikTok’s Content Moderation Allegedly Failed to Adequately Take down Videos Promoting Eating Disorders

AIID-132 · 2020-12-27 real-world incident

Class
Model Evasion (filed by a model that read the source)
Technique
Real-world incidents
Affected
Tiktok (incident)

What happened, in short

Users deliberately used common misspellings of banned search terms, which evaded TikTok's automated detection and let eating disorder videos stay up. (Our summary, written by a model. The source text is below.)

Details

Videos promoting eating disorders evaded TikTok's automated violation detection system without difficulty via common misspellings of search terms, bypassing its ban of violating hashtags such as "proana" and "anorexia".

References

Source: AI Incident Database, used under CC BY-SA 4.0.