Machine Learning as an Adversarial Service: Learning Black-Box Adversarial Examples

By Jamie Hayes, George Danezis. 2017

56 citations recorded by the source corpus.

Neural networks are known to be vulnerable to adversarial examples, inputs that have been intentionally perturbed to remain visually similar to the source input, but cause a misclassification. Until now, black-box attacks against neural networks have relied on transferability of adversarial examples. White-box attacks are used to generate adversarial examples on a substitute model and then transferred to the black-box target model. In this paper, we introduce a direct attack against black-box neural networks, that uses another attacker neural network to learn to craft adversarial examples. We show that our attack is capable of crafting adversarial examples that are indistinguishable from the source input and are misclassified with overwhelming probability - reducing accuracy of the black-box neural network from 99.4% to 0.77% on the MNIST dataset, and from 91.4% to 6.8% on the CIFAR-10 dataset. Our attack can adapt and reduce the effectiveness of proposed defenses against adversarial examples, requires very little training data, and produces adversarial examples that can transfer to different machine learning models such as Random Forest, SVM, and K-Nearest Neighbor. To demonstrate the practicality of our attack, we launch a live attack against a target black-box model hosted online by Amazon: the crafted adversarial examples reduce its accuracy from 91.8% to 61.3%. Additionally, we show attacks proposed in the literature have unique, identifiable distributions. We use this information to train a classifier that is robust against such attacks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Adversarial examples are malicious test inputs obtained by perturbing legitimate inputs [17], [23], [64], [65], [66], [67], [68], [69], [70].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “com [53] services due to the transferability of adversarial examples proposed by Papernot et al.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, methodology): “Along the same direction, Hayex and Danezis [47] also used an attacker neural network to learn adversarial examples for black-box attacks.”
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples (2018, background): “Since then, a number of approaches have surfaced regarding adversarial attacks on CNNs, such as using gradient-based methods [2, 11, 12, 13, 14, 15], score-based methods [16, 17, 18], transfer-based methods [19], and decision-based methods [20].”
Attacks on Machine Learning: Lurking Danger for Accountability (2019, methodology): “Attack [4] k-Nearest Neighbors Supervised Yes/No Poisoning Attack [46] No Adversarial Examples [31] k-Means Clustering Unsupervised No Ateniese et al.”
Towards Reverse-Engineering Black-Box Neural Networks (2017, methodology): “(2) Use the avatar approach to train a white box network that is supposedly similar to the target (Papernot et al., 2016b;a; Hayes & Danezis, 2017).”
Query-Efficient Black-box Adversarial Examples (2017, background): “Prior work has demonstrated that black-box methods can feasibly attack real-world, commercially deployed systems, including image classification APIs from Clarifai, Metamind, Google, and Amazon [10, 14, 8], and a speech recognition system from Google [2].”
Increasing the robustness of DNNs against image corruptions by playing the Game of Noise (2020, background): “Hayes and Danezis [2017] generate UAPs by training so-called universal adversarial networks (UANs).”
Black-box Adversarial Attacks with Limited Queries and Information (2018, methodology): “In this paper, we use the deﬁnition of black-box access as query access (Chen et al., 2017; Liu et al., 2017; Hayes & Danezis, 2017).”