INCIDENT MAP

Browse / Model Evasion / AML.CS0004

Camera Hijack Attack on Facial Recognition System

AML.CS0004 · 2020-01-01 case study

Class
Model Evasion
Technique
Evade AI Model
Also involves
Misinformation and External Harm
Target
Shanghai government tax office's facial recognition service (Incident)

How it worked

  1. Reconnaissance · Gather Victim Identity Information

    The attackers collected user identity information and high-definition face photos from an online black market.

  2. Resource Development · Establish Accounts

    The attackers used the victim identity information to register new accounts in the tax system.

  3. Resource Development · Acquire Infrastructure: Consumer Hardware

    The attackers bought customized low-end mobile phones.

  4. Resource Development · Obtain Capabilities: Software Tools

    The attackers obtained customized Android ROMs and a virtual camera application.

  5. Resource Development · Obtain Capabilities: Adversarial AI Attack Implementations

    The attackers obtained software that turns static photos into videos, adding realistic effects such as blinking eyes.

  6. AI Model Access · AI-Enabled Product or Service

    The attackers used the virtual camera app to present the generated video to the ML-based facial recognition service used for user verification.

  7. Initial Access · Evade AI Model

    The attackers successfully evaded the face recognition system. This allowed the attackers to impersonate the victim and verify their identity in the tax system.

  8. Impact · External Harms: Financial Harm

    The attackers used their privileged access to the tax system to send invoices to supposed clients and further their fraud scheme.

Summary

This type of camera hijack attack can evade the traditional live facial recognition authentication model and enable access to privileged systems and victim impersonation.

Two individuals in China used this attack to gain access to the local government's tax system. They created a fake shell company and sent invoices via tax system to supposed clients. The individuals started this scheme in 2018 and were able to fraudulently collect $77 million.

References

Source: MITRE ATLAS case studies, used under Apache 2.0.