Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition

By Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter. 2016

1,803 citations recorded by the source corpus.

Machine learning is enabling a myriad innovations, including new algorithms for cancer diagnosis and self-driving cars. The broad use of machine learning makes it important to understand the extent to which machine-learning algorithms are subject to attack, particularly when used in applications where physical security or safety is at risk. In this paper, we focus on facial biometric systems, which are widely used in surveillance and access control. We define and investigate a novel class of attacks: attacks that are physically realizable and inconspicuous, and allow an attacker to evade recognition or impersonate another individual. We develop a systematic method to automatically generate such attacks, which are realized through printing a pair of eyeglass frames. When worn by the attacker whose image is supplied to a state-of-the-art face-recognition algorithm, the eyeglasses allow her to evade being recognized or to impersonate another individual. Our investigation focuses on white-box face-recognition systems, but we also demonstrate how similar techniques can be used in black-box scenarios, as well as to avoid face detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Natural Adversarial Examples (2019, background): “luding elastic deformations [43], adversarial coloring [3,23], and synthesis via generative models [2,39] and evolutionary search [35], among others. Other work has shown how to print 2D [32,5] or 3D [38,1] objects that fool classiﬁers. However, these existing adversarial attacks are all based on sy”
Towards the Science of Security and Privacy in Machine Learning (2016, background): “Sharif et al. applied the approach introduced in [30] to ﬁnd adversarial examples that are printed on glasses frames, which once worn by an individual result in its face being misclassiﬁed by a face recognition model [37].”
Towards Deep Learning Models Resistant to Adversarial Attacks (2017, background): “Computer vision presents a particularly striking challenge: very small changes to the input image can fool state-of-the-art neural networks with high probability [23, 7, 15, 21, 14].”
Unsolved Problems in ML Safety (2021, background): “If the adversary wears that speciﬁc item of jewelry, the backdoored facial recognition will allow the adversary into the building [160].”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, background): “Despite the excellent performance of CNNs on object recognition, the vulnerability to adversarial attacks suggests that superficial changes can result in highly non-human shifts in prediction [43, 35, 26, 1, 15, 41, 3, 40].”
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “This wrong start date is implicitly acknowledged in a growing number of recent papers in the area of computer security [5, 7, 10, 12–16] and computer vision [8, 9, 17], which only consider recent work on the security of deep networks against adversarial examples, almost completely ignoring previous…”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, methodology): “More speciﬁcally we consider an adversary that makes use of localized universal malicious objects, e.g., printed patches [26], [3], [10], [9] or 3D objects [2] (see, e.g., Figure 1), that can fool a model under real-world conditions such as lighting, sensor noise, and rotation.”
Spectral Signatures in Backdoor Attacks (2018, background): “We also point out that backdoor poisoning is related to adversarial examples [13, 26, 21, 12, 29, 4, 24, 32].”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “85, which is the same as previous work [59].”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, methodology): “There is a long line of work (Szegedy et al., 2013; Goodfellow et al., 2014; Kurakin et al., 2016a;b; Sharif et al., 2016; Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2016; Papernot et al., 2017; Madry et al., 2017; Athalye et al., 2017) on methods for constructing adversarial perturbations Equ”
STRIP: a defence against trojan attacks on deep neural networks (2019, background): “One distinctive feature of trojan attacks on vision systems is that they are physically realizable [11]–[13].”
Stealing Hyperparameters in Machine Learning (2018, background): “Evasion attacks: In these attacks [33], [22], [3], [51], [52], [17], [26], [23], [37], [56], [9], [44], [28], [36], an attacker aims to inject carefully crafted noise into a testing instance (e.g., an email spam, a social spam, a malware, or a face image) such that the classiﬁer predicts a…”
Certified Defenses against Adversarial Examples (2018, background): “In addition to being an intriguing phenonemon, the existence of such “adversarial examples” exposes a serious vulnerability in current ML systems (Evtimov et al., 2017; Sharif et al., 2016; Carlini et al., 2016).”