CASEBOOK

Browse / Unsafe Model Loading and Supply Chain / AIID-731

Purportedly Hallucinated Software Packages with Potential Malware Reportedly Downloaded Thousands of Times by Developers

AIID-731 ยท 2023-12-01 real-world incident

Class
Unsafe Model Loading and Supply Chain (filed by a model that read the source)
Technique
Real-world incidents
Affected
Openai, Meta, Google (incident)

What happened, in short

Attackers registered package names that LLMs had hallucinated, and developers downloaded the malicious packages over 15,000 times, introducing potential malware into codebases through slopsquatting. (Our summary, written by a model. The source text is below.)

Details

Large language models have reportedly hallucinated non-existent software package names, some of which were subsequently uploaded to public repositories and incorporated into real codebases. In one case, a package named huggingface-cli, which was purported to have been originally suggested by an AI model, was downloaded more than 15,000 times. This dynamic enables what security researchers have termed "slopsquatting," in which attackers register hallucinated package names and introduce potential malware into software supply chains.

References

Source: AI Incident Database, used under CC BY-SA 4.0.