INCIDENT MAP

Browse / Direct Prompt Injection / AML.CS0037

Data Exfiltration via Agent Tools in Copilot Studio

AML.CS0037 · 2025-06-01 case study

Class
Direct Prompt Injection
Technique
LLM Prompt Crafting
Also involves
MCP and Tool Server Flaws, Sensitive Data Disclosure, Indirect Prompt Injection
Target
Copilot Studio Customer Service Agent (Exercise)

How it worked

  1. Reconnaissance · Active Scanning: Probe AI Agent Trigger Channels

    The researchers look for support email addresses on the target organization's website which may be managed by an AI agent. Then, they probe the system by sending emails and looking for indications of agentic AI in automatic replies.

  2. AI Attack Adaptation · LLM Prompt Crafting

    Once a target has been identified, the researchers craft prompts designed to probe for a potential AI agent monitoring the inbox. The prompt instructs the agent to send an email reply to an address of the researchers' choosing.

  3. Initial Access · Prompt Infiltration via Public-Facing Application

    The researchers send an email with the malicious prompt to the inbox they suspect may be managed by an AI agent.

  4. Execution · LLM Prompt Injection: Triggered

    The researchers receive a reply at the address they specified, indicating that there is an AI agent present, and that the triggered prompt injection was successful.

  5. Discovery · Discover AI Agent Configuration: Activation Triggers

    The researchers infer that the AI agent is activated when receiving an email.

  6. Discovery · Discover AI Agent Configuration: Tool Definitions

    The researchers infer that the AI agent has a tool for sending emails.

  7. AI Model Access · AI-Enabled Product or Service

    From here, the researchers repeat the same steps to interact with the AI agent, sending malicious prompts to the agent via email and receiving responses at their desired address.

  8. Execution · LLM Prompt Injection

    The researchers modify the original prompt to discover other knowledge sources and tools that may have data they are after.

  9. Discovery · Discover AI Agent Configuration: Embedded Knowledge

    The researchers discover the AI agent has access to a "Customer Support Account Owners.csv" data source.

  10. Discovery · Discover AI Agent Configuration: Tool Definitions

    The researchers discover the AI agent has access to the Salesforce get-records tool, which can be used to retrieve CRM records.

  11. AI Attack Adaptation · LLM Prompt Crafting

    The researchers put their knowledge of the AI agent's tools and knowledge sources together to craft a prompt that will collect and exfiltrate the customer data they are after.

  12. Collection · Data from AI Services: RAG Databases

    The prompt asks the agent to retrieve all of the fields and rows from "Customer Support Account Owners.csv". The agent retrieves the entire file.

  13. Collection · Data from AI Services: AI Agent Tools

    The prompt asks the agent to retrieve all Salesforce records using its get-records tool. The agent retrieves all records from the victim's CRM.

  14. Exfiltration · Exfiltration via AI Agent Tool Invocation

    The prompt asks the agent to email the results to an address of the researcher's choosing using its email tool. The researchers successfully exfiltrate their target data via the tool invocation.

Summary

Researchers from Zenity demonstrated how an organization's data can be exfiltrated via prompt injections that target an AI-powered customer service agent.

The target system is a customer service agent built by Zenity in Copilot Studio. It is modeled after an agent built by McKinsey to streamline its customer service needs. The AI agent listens to a customer service email inbox where customers send their engagement requests. Upon receiving a request, the agent looks at the customer's previous engagements, understands who the best consultant for the case is, and proceeds to send an email to the respective consultant regarding the request, including all of the relevant context the consultant will need to properly engage with the customer.

The Zenity researchers begin by performing targeting to identify an email inbox that is managed by an AI agent. Then they use prompt injections to discover details about the AI agent, such as its knowledge sources and tools. Once they understand the AI agent's capabilities, the researchers are able to craft a prompt that retrieves private customer data from the organization's RAG database and CRM, and exfiltrate it via the AI agent's email tool.

Vendor Response: Microsoft quickly acknowledged and fixed the issue. The prompts used by the Zenity researchers in this exercise no longer work, however other prompts may still be effective.

References

Source: MITRE ATLAS case studies, used under Apache 2.0.