Browse / Model Evasion / AML.CS0008
ProofPoint Evasion
- Class
- Model Evasion
- Also involves
- Model Theft and Extraction
- Target
- ProofPoint Email Protection System (Exercise)
How it worked
- Discovery · Discover AI Model Outputs
The researchers discovered that ProofPoint's Email Protection left model output scores in email headers.
- Collection · Outputs from AI Models
The researchers sent many emails through the system to collect model outputs from the headers.
- AI Attack Adaptation · Replicate AI Model Behavior: Replicate Behavior from Model Outputs
The researchers used the emails and collected scores as a dataset, which they used to train a functional copy of the ProofPoint model.
Basic correlation was used to decide which score variable speaks generally about the security of an email. The "mlxlogscore" was selected in this case due to its relationship with spam, phish, and core mlx and was used as the label. Each "mlxlogscore" was generally between 1 and 999 (higher score = safer sample). Training was performed using an Artificial Neural Network (ANN) and Bag of Words tokenizing.
- AI Attack Adaptation · Craft Adversarial Data: Black-Box Transfer
Next, the ML researchers algorithmically found samples from this "offline" proxy model that helped give desired insight into its behavior and influential variables.
Examples of good scoring samples include "calculation", "asset", and "tyson".
Examples of bad scoring samples include "software", "99", and "unsub". - Impact · Evade AI Model
Finally, these insights from the "offline" proxy model allowed the researchers to create malicious emails that received preferable scores from the real ProofPoint email protection system, hence bypassing it.
Summary
Proof Pudding (CVE-2019-20634) is a code repository that describes how ML researchers evaded ProofPoint's email protection system by first building a copy-cat email protection ML model, and using the insights to bypass the live system. More specifically, the insights allowed researchers to craft malicious emails that received preferable scores, going undetected by the system. Each word in an email is scored numerically based on multiple variables and if the overall score of the email is too low, ProofPoint will output an error, labeling it as SPAM.
References
- https://nvd.nist.gov/vuln/detail/CVE-2019-20634
- https://github.com/moohax/Talks/blob/master/slides/DerbyCon19.pdf
- https://github.com/moohax/Proof-Pudding
- https://www.youtube.com/watch?v=CsvkYoxtexQ&ab-channel=AdrianCrenshaw
Source: MITRE ATLAS case studies, used under Apache 2.0.