INCIDENT MAP

Browse / Model Theft and Extraction / AML.CS0075

Model Distillation Campaigns Targeting Google AI Services

AML.CS0075 · 2025-01-01 case study

Class
Model Theft and Extraction
Technique
Exfiltration via AI Inference API
Also involves
Direct Prompt Injection, Misinformation and External Harm, AI Framework and Platform Flaws
Target
Google AI Services, including Gemini (Incident)

How it worked

  1. Resource Development · Compromise Accounts

    Unknown actors compromised user credentials for Google AI services.

  2. Resource Development · Establish Accounts

    Unknown actors established many fraudulent accounts for Google AI Services.

  3. Resource Development · Acquire Infrastructure: AI Service Proxies

    Unknown actors deployed proxy infrastructure to orchestrate large-scale automated activity targeting Google AI models. The infrastructure supported requests distributed across credentials, accounts, and product channels.

  4. Initial Access · Valid Accounts

    The fraudulent and compromised accounts were used to access Google AI services via the proxy infrastructure.

  5. Defense Evasion · Centralized AI Service Request Routing

    Unknown actors rotated queries across thousands of valid accounts across different product channels to obscure their origin and bypass security controls.

  6. AI Attack Adaptation · LLM Prompt Crafting

    Unknown actors crafted prompts to elicit outputs supporting capability replication.

  7. Exfiltration · Exfiltration via AI Inference API

    Unknown actors used the Google service inference APIs to acquire model outputs.

  8. Collection · Outputs from AI Models

    Unknown actors collected the model responses as training material for distillation.

    Google identified over 100,000 prompts in the February 2026 reasoning-focused campaign. Later campaigns targeting reasoning and other capabilities exceeded 100 million prompts

  9. Resource Development · Replicate AI Model Behavior: Replicate Behavior from Model Outputs

    The distillation campaigns used outputs for training or fine-tuning models to reproduce selected Google model capabilities.

  10. Impact · External Harms: AI Intellectual Property Theft

    The campaigns sought to appropriate proprietary capabilities from Google's models through distillation, reducing the effort and expense required to develop comparable capabilities independently.

Summary

Google Threat Intelligence Group and Google DeepMind reported recurring distillation campaigns targeting Google AI services. Adversaries sought model outputs to reproduce capabilities, including reasoning, visual and audio understanding, image generation, and video generation.

The unknown actors utilized proxy infrastructure to orchestrate requests, distributing queries across thousands of compromised and fraudulent accounts spanning different product channels. Google reported that this fragmented activity obscured the attackers' origin and helped bypass security controls.[[google-sep]] The actors queried Google's services to elicit responses and collected the outputs. Model outputs are subsequently used by the actors to train or fine-tune their own models.

In February 2026, Google reported a campaign involving over 100,000 prompts that attempted to obtain Gemini's full reasoning traces rather than the summaries normally delivered to users.[[google-feb]] By September 2026, Google reported regularly observing coordinated distillation campaigns, some exceeding 100 million prompts. These campaigns targeted reasoning as well as multimodal capabilities, including visual and audio understanding, image generation, and video generation.[[google-sep]]

Google reported recognizing the reasoning-focused attack in real time and reducing its risk while protecting internal reasoning traces. For the broader campaigns, Google reported measures to reduce their utility and block responsible accounts, as well as techniques to identify Gemini-distilled models and trace their provenance.[[google-feb]][[google-sep]]

References

Source: MITRE ATLAS case studies, used under Apache 2.0.