INCIDENT MAP

Browse / Indirect Prompt Injection / AML.CS0066

ZombieAgent: Data Exfiltration Attack on ChatGPT

AML.CS0066 · 2025-09-25 case study

Class
Indirect Prompt Injection
Technique
Prompt Infiltration via Public-Facing Application
Also involves
Jailbreak and Guardrail Bypass, Sensitive Data Disclosure, Tool and Agent Misuse
Target
OpenAI ChatGPT (Exercise)

How it worked

  1. AI Attack Adaptation · LLM Prompt Crafting

    The researchers crafted malicious prompt payloads for the different attack variants. The payloads contained instructions for connector access, data collection, static-URL encoding, memory manipulation, and propagation.

  2. Resource Development · Stage Capabilities

    The researchers prepared infrastructure to receive exfiltrated data.

  3. Defense Evasion · LLM Prompt Obfuscation

    The prompt injection was visually concealed in externally controlled content using techniques such as white-on-white text or microscopic font sizes. ChatGPT could process the instructions even though they were not apparent to the user.

  4. Initial Access · Prompt Infiltration via Public-Facing Application

    The researchers sent a malicious email to an inbox connected to ChatGPT or provided a malicious document that could be uploaded or retrieved through a connected service. This introduced the prompt into a data source accessible to the victim's agent.

  5. Execution · LLM Prompt Injection: Indirect

    A later, legitimate user request, such as asking ChatGPT to summarize the inbox, caused the agent to retrieve the malicious content and execute the concealed instructions. The user did not knowingly interact with the malicious email.

  6. Persistence · AI Agent Context Poisoning: Memory

    The malicious instructions caused ChatGPT to create or modify memories. The poisoned memories instructed ChatGPT to retain sensitive information from conversations and to perform attacker-defined actions during later interactions.

  7. Privilege Escalation · AI Agent Tool Invocation

    The malicious instructions caused ChatGPT to invoke connector and web-access capabilities available under the victim's authority. This gave the prompt access to information and actions that were not directly available to the researchers.

  8. Collection · Data from AI Services: AI Agent Tools

    ChatGPT used connected-service tools to retrieve information accessible to the victim's agent. Demonstrated collection included mailbox content and email contact information.

  9. Defense Evasion · LLM Jailbreak

    The researchers bypassed ChatGPT's restriction against dynamically constructing or modifying URLs by supplying an indexed dictionary of static URLs. ChatGPT encoded collected data by selecting and opening the URL corresponding to each character and position.

  10. Exfiltration · Exfiltration via AI Agent Tool Invocation

    ChatGPT invoked its URL-opening capability to request the selected adversary-controlled URLs. The researchers reconstructed the sensitive data from the character and position encoded in the resulting server requests. The requests originated from OpenAI's infrastructure rather than the victim's endpoint or corporate network.

  11. Collection · Data from AI Services: AI Agent Tools

    ChatGPT searched the victim's mailbox and collected email addresses belonging to potential additional targets.

  12. Lateral Movement · LLM Prompt Propagation

    The malicious instructions caused ChatGPT to reproduce the prompt in new emails or documents and distribute them to collected contacts. If another AI agent processed the poisoned content, the attack could propagate between users or connected AI systems.

Summary

ZombieAgent is a proof-of-concept indirect prompt injection attack demonstrated by Radware against OpenAI's ChatGPT Deep Research and Connector functionality. The attacks showed how instructions concealed in externally controlled content, such as emails and documents, could be ingested and executed by ChatGPT during normal user activity. The demonstration used Gmail as the injection source, but any ChatGPT Connector such as Outlook, Google Drive, Jira, or Teams could be similarly abused.

The Radware Security Researchers sent a malicious email containing concealed instructions to a Gmail inbox connected to ChatGPT. When the user later asked ChatGPT to perform an ordinary inbox-related task, ChatGPT retrieved the email and executed its instructions. The user did not open, click, or knowingly interact with the malicious email.

The injected instructions caused ChatGPT to collect information from connected services and exfiltrate it through URL requests. OpenAI had introduced a control preventing ChatGPT from dynamically constructing or modifying URLs which could be used to exfiltrate data via query parameters. The researchers bypassed this control by supplying an indexed dictionary of preconstructed static URLs and instructing ChatGPT to open URLs corresponding to individual characters to exfiltrate the collected data.

The researchers also showed that the malicious instructions could manipulate ChatGPT's Memory. The injected memories instructed ChatGPT to retain sensitive information from future conversations and to retrieve and execute a designated attacker-controlled email during later interactions. This created a persistent mechanism for repeated collection and exfiltration across chat sessions.

The researchers also demonstrated how the malicious prompt could be propagated. A compromised agent could collect email addresses from the victim's mailbox and use its connected email capabilities to send additional messages containing the malicious prompt to those contacts. Recipients whose AI agents later processed the poisoned messages could become additional victims.

References

Source: MITRE ATLAS case studies, used under Apache 2.0.