Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

By Xiaoyu Cao, Neil Zhenqiang Gong. 2017

219 citations recorded by the source corpus.

Deep neural networks (DNNs) have transformed several artificial intelligence research areas including computer vision, speech recognition, and natural language processing. However, recent studies demonstrated that DNNs are vulnerable to adversarial manipulations at testing time. Specifically, suppose we have a testing example, whose label can be correctly predicted by a DNN classifier. An attacker can add a small carefully crafted noise to the testing example such that the DNN classifier predicts an incorrect label, where the crafted testing example is called adversarial example. Such attacks are called evasion attacks. Evasion attacks are one of the biggest challenges for deploying DNNs in safety and security critical applications such as self-driving cars. In this work, we develop new DNNs that are robust to state-of-the-art evasion attacks. Our key observation is that adversarial examples are close to the classification boundary. Therefore, we propose region-based classification to be robust to adversarial examples. Specifically, for a benign/adversarial testing example, we ensemble information in a hypercube centered at the example to predict its label. In contrast, traditional classifiers are point-based classification, i.e., given a testing example, the classifier predicts its label based on the testing example alone. Our evaluation results on MNIST and CIFAR-10 datasets demonstrate that our region-based classification can significantly mitigate evasion attacks without sacrificing classification accuracy on benign examples. Specifically, our region-based classification achieves the same classification accuracy on testing benign examples as point-based classification, but our region-based classification is significantly more robust than point-based classification to state-of-the-art evasion attacks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, neutral): “, [10,25,26,38,41,48,64]) were explored to mitigate them.”
Stealing Hyperparameters in Machine Learning (2018, background): “Cao and Gong [8] found that adversarial examples, especially those generated by the attacks proposed by Carlini and Wagner, are close to the classification boundary.”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “Several works (Liu et al., 2018; Cao & Gong, 2017) proposed randomized smoothing as a heuristic defense, but did not prove any guarantees.”
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers (2019, background): “Randomized smoothing was proposed by several works [23, 3] as a heuristic defense without proving any guarantees.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, methodology): “Their method builds on the notion of randomized smoothing introduced in [416] and [309].”
Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification (2018, result): “Note that these results are slightly different from those that we reported in our conference paper [2].”
LEMNA: Explaining Deep Learning based Security Applications (2018, unknown): “To improve the model resistance, researchers have proposed various defense methods [9, 20, 36, 40, 67].”
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary (2018, methodology): “The relationship between an adversarial attack and a decision boundary was used to prevent an adversarial attack in Cao and Gong (2017).”
Attack Strength vs. Detectability Dilemma in Adversarial Machine Learning (2018, background): “The research into defending deep learning models can be split into two primary approaches, detecting adversarial examples and building robust classifiers [28].”
Retrospective Adversarial Replay for Continual Learning (2022, background): “Training on such samples can help the model to learn about the boundaries between previous and current tasks and also retain the knowledge of previous tasks [7, 11, 23].”
HopSkipJumpAttack: A Query-Efficient Decision-Based Attack (2019, methodology): “Multiple variants have been proposed to randomize the gradients [35–39].”
On Adaptive Attacks to Adversarial Example Defenses (2020, background): “An idea related to this term alone has previously been suggested as a defense (Cao & Gong, 2017) and was subsequently broken (He et al., 2018).”
Efficient Neural Network Robustness Certification with General Activation Functions (2018, background): “Some recent works (such as robust optimization based adversarial training [29] or region-based classification [30]) empirically exhibit strong robustness against several adversarial attacks, which is beyond the scope of provable robustness certification.”
AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning (2018, methodology): “We consider robust classifiers based on adversarial training [22], defensive distillation [44], and region-based classification [45].”
ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models (2018, background): “Besides the above, many recent works have tackled security and privacy in machine learning from various perspectives [39], [16], [9], [37], [7], [27], [20], [45], [36], [21], [40], [29], [41], [20].”
Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020, unknown): “-based classfication [217] no no no yes no JPEG compression [218] no no yes no no”
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples (2019, background): “, adversarial training [19, 35, 63], defensive distillation [51], Region-based Classification [9], MagNet [37], and Feature Squeezing [68]) have been explored to make classifiers robust against adversarial examples, it is still considered an open challenge to design such robust classifiers.”