INCIDENT MAP

Browse / Model Evasion / AML.CS0010

Microsoft Azure Service Disruption

AML.CS0010 · 2020-01-01 case study

Class
Model Evasion
Technique
Craft Adversarial Data: White-Box Optimization
Also involves
Model Theft and Extraction, AI Framework and Platform Flaws
Target
Internal Microsoft Azure Service (Exercise)

How it worked

  1. Reconnaissance · Search Open Technical Databases

    The team first performed reconnaissance to gather information about the target ML model.

  2. Initial Access · Valid Accounts

    The team used a valid account to gain access to the network.

  3. Collection · AI Artifact Collection

    The team found the model file of the target ML model and the necessary training data.

  4. Exfiltration · Exfiltration via Cyber Means

    The team exfiltrated the model and data via traditional means.

  5. AI Attack Adaptation · Craft Adversarial Data: White-Box Optimization

    Using the target model and data, the red team crafted evasive adversarial data in an offline manner.

  6. AI Model Access · AI Model Inference API Access

    The team used an exposed API to access the target model.

  7. AI Attack Adaptation · Verify Attack

    The team submitted the adversarial examples to the API to verify their efficacy on the production system.

  8. Impact · Evade AI Model

    The team performed an online evasion attack by replaying the adversarial examples and accomplished their goals.

Summary

The Microsoft AI Red Team performed a red team exercise on an internal Azure service with the intention of disrupting its service. This operation had a combination of traditional ATT&CK enterprise techniques such as finding valid account, and exfiltrating data -- all interleaved with adversarial ML specific steps such as offline and online evasion examples.

Source: MITRE ATLAS case studies, used under Apache 2.0.