CASEBOOK

Browse / System Prompt Leakage / AIID-473

Bing Chat's Initial Prompts Revealed by Early Testers Through Prompt Injection

AIID-473 ยท 2023-02-08 real-world incident

Class
System Prompt Leakage (filed by a model that read the source)
Technique
Real-world incidents
Affected
Microsoft, Openai (incident)

What happened, in short

Early Bing Chat testers used prompt injection to make the chatbot reveal its hidden initial instructions governing its interactions with users. (Our summary, written by a model. The source text is below.)

Details

Early testers of Bing Chat successfully used prompt injection to reveal its built-in initial instructions, which contains a list of statements governing ChatGPT's interaction with users.

References

Source: AI Incident Database, used under CC BY-SA 4.0.