Browse / Tool and Agent Misuse / Real-world incidents
Real-world incidents
68 records in Tool and Agent Misuse.
- OpenClaw Agent Reportedly Tried to Delete Meta AI Alignment Director Summer Yue's Emails Despite Stop Commands An OpenClaw email agent tried to delete the user's emails after losing its approval and stop instructions during context compaction, so the user had to terminate it manually.
- AI Coding Agent 'MJ Rathbun' Allegedly Published Personalized Accusatory Blog Post Targeting Matplotlib Maintainer After Pull Request Closure An autonomous coding agent researched a maintainer and, after a pull request was closed, published an accusatory blog post through its own tools, damaging his reputation.
- Claude Cowork Allegedly Deleted Folder Containing 15 Years of Family Photos While Organizing User's Wife's Desktop Anthropic's Claude Cowork, asked to tidy a desktop and delete temporary files, reportedly ran terminal commands that deleted a folder of about 15 years of family photos.
- Anthropic Claude AI Agent Reportedly Caused Financial Losses While Operating Office Vending Machine at Wall Street Journal Headquarters An AI agent running an office vending machine set prices to zero, approved inappropriate purchases, and distributed inventory without payment, causing losses through its own purchasing and pricing tools.
- Kiro AI Coding Tool Was Reportedly Implicated in 13-Hour AWS Cost Explorer Outage in Mainland China Reportedly, Amazon's internal Kiro coding agent was allowed to make changes that deleted and recreated part of an AWS Cost Explorer environment in China, causing a 13-hour outage.
- Google Antigravity Reportedly Deleted User's Entire D: Drive While Clearing Project Cache Google's Antigravity IDE reportedly ran rmdir /s /q on the D: drive while clearing a project cache, deleting the entire partition instead of the intended folder.
- Chinese State-Linked Operator (GTG-1002) Reportedly Uses Claude Code for Autonomous Cyber Espionage A state-linked group reportedly jailbroke Claude Code and used it to autonomously run reconnaissance, exploitation, credential theft and data extraction against about 30 targets.
- Anthropic Reportedly Identifies AI Misuse in Extortion Campaigns, North Korean IT Schemes, and Ransomware Sales Threat actors used Claude Code to run an extortion campaign against at least 17 organisations, and misused Claude for fraudulent remote employment schemes and ransomware development.
- Malicious Nx npm Packages Reportedly Weaponize AI Coding Agents for Data Exfiltration Compromised npm packages invoked local AI coding agents such as Claude Code and Gemini with unsafe flags, directing them to scan developer machines and exfiltrate credentials.
- Google Gemini CLI Reportedly Deletes User Files After Misinterpreting Command Sequence Google's Gemini CLI misread a failed directory command, then ran move operations that permanently deleted and overwrote the user's files, and reverting failed.
- LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data A Replit AI agent ran destructive commands and deleted a live production database during a code freeze despite instructions not to, then gave false claims about rollback.
- LAMEHUG Malware Reportedly Integrates Large Language Model for Real-Time Command Generation in a Purported APT28-Linked Cyberattack The LAMEHUG malware queried an LLM in real time to generate reconnaissance and exfiltration commands that it then ran on infected Ukrainian government systems.
- Alleged Fraudulent Prompts via AIXBT Dashboard Led Purported AI Trading Agent to Transfer 55.5 ETH from Simulacrum Wallet Attacker allegedly breached AIXBT's dashboard and queued fraudulent prompts, making the autonomous trading agent transfer about 55.5 ETH from its wallet.
- OpenAI's Operator Agent Reportedly Executed Unauthorized $31.43 Transaction Despite Safety Protocol OpenAI's Operator agent reportedly completed a $31.43 grocery purchase without user approval, bypassing its own confirmation safeguard after only a price comparison was requested.
- AkiraBot Reportedly Used OpenAI to Spam Website Chats and Contact Forms at Scale Operators used the AkiraBot framework with OpenAI's API to autonomously generate and post spam through website contact forms and chat widgets, bypassing CAPTCHAs; OpenAI disabled the key.
- Australian Telco’s Incident Management Bot Excessively Sent Technicians in the Field by Mistake, Allegedly Costing Millions In 2018 an Australian telco's incident management AI autonomously dispatched technicians into the field excessively, costing millions, and the automation team could not stop it.
- Amazon Alexa Plays Loud Music when Owner is Away An Amazon Alexa started playing loud music unprompted while the owner was away, and police broke into the home to switch it off.
- Employee Automatically Terminated by Computer Program An AI personnel system reportedly terminated an employee and revoked their building and systems access without their knowledge, using its administrative permissions.