Browse / Model Theft and Extraction / AML.CS0007
GPT-2 Model Replication
- Target
- OpenAI GPT-2 (Exercise)
How it worked
- Reconnaissance · Search Open Technical Databases
Using the public documentation about GPT-2, the researchers gathered information about the dataset, model architecture, and training hyper-parameters.
- Resource Development · Acquire Public AI Artifacts: Models
The researchers obtained a reference implementation of a similar publicly available model called Grover.
- Resource Development · Acquire Public AI Artifacts: Datasets
The researchers were able to manually recreate the dataset used in the original GPT-2 paper using the gathered documentation.
- Resource Development · Acquire Infrastructure: AI Development Workspaces
The researchers were able to use TensorFlow Research Cloud via their academic credentials.
- AI Attack Adaptation · Replicate AI Model Behavior: Replicate Behavior from Gathered AI Artifacts
The researchers modified Grover's objective function to reflect GPT-2's objective function and then trained on the dataset they curated using used Grover's initial hyperparameters. The resulting model functionally replicates GPT-2, obtaining similar performance on most datasets.
A bad actor who followed the same procedure as the researchers could then use the replicated GPT-2 model for malicious purposes.
Summary
OpenAI built GPT-2, a language model capable of generating high quality text samples. Over concerns that GPT-2 could be used for malicious purposes such as impersonating others, or generating misleading news articles, fake social media content, or spam, OpenAI adopted a tiered release schedule. They initially released a smaller, less powerful version of GPT-2 along with a technical description of the approach, but held back the full trained model.
Before the full model was released by OpenAI, researchers at Brown University successfully replicated the model using information released by OpenAI and open source ML artifacts. This demonstrates that a bad actor with sufficient technical skill and compute resources could have replicated GPT-2 and used it for harmful goals before the AI Security community is prepared.
References
- https://www.wired.com/story/dangerous-ai-open-source/
- https://blog.usejournal.com/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc
Source: MITRE ATLAS case studies, used under Apache 2.0.