Browse / Model Evasion / AML.CS0014
Confusing Antimalware Neural Networks
- Class
- Model Evasion
- Also involves
- Model Theft and Extraction
- Target
- Kaspersky's Antimalware ML Model (Exercise)
How it worked
- Reconnaissance · Search Open AI Vulnerability Analysis
The researchers performed a review of adversarial ML attacks on antimalware products.
They discovered that techniques borrowed from attacks on image classifiers have been successfully applied to the antimalware domain.
However, it was not clear if these approaches were effective against the ML component of production antimalware solutions. - Reconnaissance · Search Victim-Owned Websites
Kaspersky's use of ML-based antimalware detectors is publicly documented on their website. In practice, an adversary could use this for targeting.
- AI Model Access · AI-Enabled Product or Service
The researchers used access to the target ML-based antimalware product throughout this case study.
This product scans files on the user's system, extracts features locally, then sends them to the cloud-based ML malware detector for classification.
Therefore, the researchers had only black-box access to the malware detector itself, but could learn valuable information for constructing the attack from the feature extractor. - Collection · Outputs from AI Models
The researchers collected a dataset of malware and clean files.
They scanned the dataset with the target ML-based antimalware solution and labeled the samples according to the ML detector's predictions. - AI Attack Adaptation · Replicate AI Model Behavior
A proxy model was trained on the labeled dataset of malware and clean files.
The researchers experimented with a variety of model architectures. - Resource Development · Develop Capabilities: Adversarial AI Attacks
By reverse engineering the local feature extractor, the researchers could collect information about the input features, used for the cloud-based ML detector.
The model collects PE Header features, section features and section data statistics, and file strings information.
A gradient based adversarial algorithm for executable files was developed.
The algorithm manipulates file features to avoid detection by the proxy model, while still containing the same malware payload - AI Attack Adaptation · Craft Adversarial Data: Black-Box Transfer
Using a developed gradient-driven algorithm, malicious adversarial files for the proxy model were constructed from the malware files for black-box transfer to the target model.
- AI Attack Adaptation · Verify Attack
The adversarial malware files were tested against the target antimalware solution to verify their efficacy.
- Defense Evasion · Evade AI Model
The researchers demonstrated that for most of the adversarial files, the antimalware model was successfully evaded.
In practice, an adversary could deploy their adversarially crafted malware and infect systems while evading detection.
Summary
Cloud storage and computations have become popular platforms for deploying ML malware detectors.
In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers.
The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models.
They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.
References
Source: MITRE ATLAS case studies, used under Apache 2.0.