Browse / Model Evasion / AML.CS0005
Attack on Machine Translation Services
- Class
- Model Evasion
- Also involves
- Model Theft and Extraction, Misinformation and External Harm
- Target
- Google Translate, Bing Translator, Systran Translate (Exercise)
How it worked
- Reconnaissance · Search Open Technical Databases
The researchers used published research papers to identify the datasets and model architectures used by the target translation services.
- Resource Development · Acquire Public AI Artifacts: Datasets
The researchers gathered similar datasets that the target translation services used.
- Resource Development · Acquire Public AI Artifacts: Models
The researchers gathered similar model architectures that the target translation services used.
- Collection · Outputs from AI Models
They abused a public facing application to query the model and produced machine translated sentence pairs as training data.
- AI Attack Adaptation · Replicate AI Model Behavior: Replicate Behavior from Model Outputs
Using these translated sentence pairs, the researchers trained a model that replicates the behavior of the target model.
- Impact · External Harms: AI Intellectual Property Theft
By replicating the model with high fidelity, the researchers demonstrated that an adversary could steal a model and violate the victim's intellectual property rights.
- AI Attack Adaptation · Craft Adversarial Data: Black-Box Transfer
The replicated models were used to generate adversarial examples that successfully transferred to the black-box translation services.
- Impact · Evade AI Model
The adversarial examples were used to evade the machine translation services by a variety of means. This included targeted word flips, vulgar outputs, and dropped sentences.
- Impact · Erode AI Model Integrity
Adversarial attacks can cause errors that cause reputational damage to the company of the translation service and decrease user trust in AI-powered services.
Summary
Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs.
A research group at UC Berkeley utilized these public endpoints to create a replicated model with near-production state-of-the-art translation quality.
Beyond demonstrating that IP can be functionally stolen from a black-box system, they used the replicated model to successfully transfer adversarial examples to the real production services.
These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.
References
- https://arxiv.org/abs/2004.15015
- https://www.ericswallace.com/imitation
- https://thehill.com/policy/international/asia-pacific/449164-google-under-fire-for-mistranslating-chinese-amid-hong-kong/
Source: MITRE ATLAS case studies, used under Apache 2.0.