INCIDENT MAP

Browse / Model Evasion / AML.CS0000

Evasion of Deep Learning Detector for Malware C&C Traffic

AML.CS0000 · 2020-01-01 case study

Class
Model Evasion
Technique
Craft Adversarial Data: Manual Modification
Also involves
Model Theft and Extraction
Target
Palo Alto Networks malware detection system (Exercise)

How it worked

  1. Reconnaissance · Search Open Technical Databases: Pre-Print Repositories

    We identified a machine learning based approach to malicious URL detection as a representative approach and potential target from the paper URLNet: Learning a URL representation with deep learning for malicious URL detection, which was found on arXiv (a pre-print repository).

  2. Resource Development · Acquire Public AI Artifacts: Datasets

    We acquired a command and control HTTP traffic dataset consisting of approximately 33 million benign and 27 million malicious HTTP packet headers.

  3. AI Attack Adaptation · Replicate AI Model Behavior

    We trained a model on the HTTP traffic dataset to use as a proxy for the target model.
    Evaluation showed a true positive rate of ~ 99% and false positive rate of ~ 0.01%, on average.
    Testing the model with a HTTP packet header from known malware command and control traffic samples was detected as malicious with high confidence (> 99%).

  4. AI Attack Adaptation · Craft Adversarial Data: Manual Modification

    We crafted evasion samples by removing fields from packet header which are typically not used for C&C communication (e.g. cache-control, connection, etc.).

  5. AI Attack Adaptation · Verify Attack

    We queried the model with our adversarial examples and adjusted them until the model was evaded.

  6. Defense Evasion · Evade AI Model

    With the crafted samples, we performed online evasion of the ML-based spyware detection model.
    The crafted packets were identified as benign with > 80% confidence.
    This evaluation demonstrates that adversaries are able to bypass advanced ML detection techniques, by crafting samples that are misclassified by an ML model.

Summary

The Palo Alto Networks Security AI research team tested a deep learning model for malware command and control (C&C) traffic detection in HTTP traffic.
Based on the publicly available paper by Le et al., we built a model that was trained on a similar dataset as our production model and had similar performance.
Then we crafted adversarial samples, queried the model, and adjusted the adversarial sample accordingly until the model was evaded.

References

Source: MITRE ATLAS case studies, used under Apache 2.0.