Browse / Model Evasion / AML.CS0010
Microsoft Azure Service Disruption
- Class
- Model Evasion
- Also involves
- Model Theft and Extraction, AI Framework and Platform Flaws
- Target
- Internal Microsoft Azure Service (Exercise)
How it worked
- Reconnaissance · Search Open Technical Databases
The team first performed reconnaissance to gather information about the target ML model.
- Initial Access · Valid Accounts
The team used a valid account to gain access to the network.
- Collection · AI Artifact Collection
The team found the model file of the target ML model and the necessary training data.
- Exfiltration · Exfiltration via Cyber Means
The team exfiltrated the model and data via traditional means.
- AI Attack Adaptation · Craft Adversarial Data: White-Box Optimization
Using the target model and data, the red team crafted evasive adversarial data in an offline manner.
- AI Model Access · AI Model Inference API Access
The team used an exposed API to access the target model.
- AI Attack Adaptation · Verify Attack
The team submitted the adversarial examples to the API to verify their efficacy on the production system.
- Impact · Evade AI Model
The team performed an online evasion attack by replaying the adversarial examples and accomplished their goals.
Summary
The Microsoft AI Red Team performed a red team exercise on an internal Azure service with the intention of disrupting its service. This operation had a combination of traditional ATT&CK enterprise techniques such as finding valid account, and exfiltrating data -- all interleaved with adversarial ML specific steps such as offline and online evasion examples.
Source: MITRE ATLAS case studies, used under Apache 2.0.