Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

By Cong Liao, Haoti Zhong, Anna Cinzia Squicciarini, Sencun Zhu, David J. Miller. 2018

353 citations recorded by the source corpus.

Deep learning models have consistently outperformed traditional machine learning models in various classification tasks, including image classification. As such, they have become increasingly prevalent in many real world applications including those where security is of great concern. Such popularity, however, may attract attackers to exploit the vulnerabilities of the deployed deep learning models and launch attacks against security-sensitive applications. In this paper, we focus on a specific type of data poisoning attack, which we refer to as a \em backdoor injection attack. The main goal of the adversary performing such attack is to generate and inject a backdoor into a deep learning model that can be triggered to recognize certain embedded patterns with a target label of the attacker's choice. Additionally, a backdoor injection attack should occur in a stealthy manner, without undermining the efficacy of the victim model. Specifically, we propose two approaches for generating a backdoor that is hardly perceptible yet effective in poisoning the model. We consider two attack settings, with backdoor injection carried out either before model training or during model updating. We carry out extensive experimental evaluations under various assumptions on the adversary model, and demonstrate that such attacks can be effective and achieve a high attack success rate (above 90%) at a small cost of model accuracy loss with a small injection rate, even under the weakest assumption wherein the adversary has no knowledge either of the original training data or the classifier model.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Can You Really Backdoor Federated Learning? (2019, background): “Under targeted attacks (often referred to as backdoor attacks) (10; 19; 17), the goal of the adversary is to ensure that the learned model behaves differently on certain targeted sub-tasks while maintaining good overall performance on the primary task.”
STRIP: a defence against trojan attacks on deep neural networks (2019, background): “Moreover, we focus on themost common trojan attackmethodology where any input image stamped with a trigger—an input-agnostic trigger—is miscalssified to a target class and the attacker is able to easily achieve a very high attack success [3, 7, 8, 10, 14, 22, 24, 34].”
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, background): “Trigger patterns may also appear in the form of natural reﬂection (Liu et al., 2020b) or human imperceptible noise (Liao et al., 2020; Li et al., 2019; Chen et al., 2019c), making them more stealthy and hard to be detected even by human inspection.”
Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, background): “Universal Adversarial Example [31, 32] Adversarial Patch [33, 34] Poisoning Attacks Poisoning Availability Attack [19, 21] Backdoor Attack [10, 11, 13]”
Hidden Trigger Backdoor Attacks (2019, methodology): “(Liao et al. 2018) proposed to use small additive perturbations (similar to standard adversarial examples (Madry et al. 2017; Goodfellow, Shlens, and Szegedy 2014; Papernot et al. 2017)) instead of a patch to trigger the attack.”
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021, methodology): “Various backdoor attack methods are developed, and most of them are based on training data poisoning (Chen et al., 2017; Liao et al., 2018; Saha et al., 2020; Liu et al., 2020; Zhao et al., 2020).”
Backdoor Learning: A Survey (2020, background): “Natural Image Recognition MNIST [83] 28× 28 60,000 10,000 [46], [8], [59], [60], [48] [52], [65], [7], [66], [67] [15], [35], [39], [45], [56] [61], [62], [73], [74], [84] [69], [72] Fashion MNIST [85] 28× 28 60,000 10,000 [63], [69]”
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation (2019, background): “5 of the 7 pixel space triggers are patch triggers and 2 of them are perturbation triggers, including both static [35] and adversarial perturbation [35] triggers.”
Anti-Backdoor Learning: Training Clean Models on Poisoned Data (2021, background): “These can be simple patterns such as a single pixel [6] and a black-white checkerboard [1], or more complex patterns such as blending backgrounds [11], natural reﬂections [12], invisible noise [13–16], adversarial patterns [17] and sample-wise patterns [18, 19].”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, background): “[180] introduced that generated perturbation masks are injected into the training samples as a backdoor to deceive the model.”
Detecting AI Trojans Using Meta Neural Analysis (2019, methodology): “A Trojaned model may be trained by injecting Trojaned inputs into the training dataset (i.e., data poisoning) [22], [14], [34].”
Defending Neural Backdoors via Generative Distribution Modeling (2019, background): “Triggers in various forms have been explored, such as targeted physical attack [5], trojaning attack [13], single-pixel attack [4], clean-label attack [14], and invisible perturbation attack [15].”
Blind Backdoors in Deep Learning Models (2020, background): “Based on poisoning attacks [2, 5, 6, 38], some backdoor attacks [10, 28, 48, 59] add mislabeled samples to the model’s training data or apply backdoor patterns to the existing training inputs [47].”
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks (2020, background): “having the following goals: 1) high attack success rate, 2) high backdoor stealthiness, and 3) low performance impact on clean test data [37].”
Safe Distillation Box (2021, background): “Liao et al. (2020) propose to generate a backdoor that is hardly perceptible yet effective, where the backdoor injection is carried out either before model training or during model updating.”
Poisoning Web-Scale Training Datasets is Practical (2023, neutral): “…[10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can…”
A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning (2019, background): “A more realistic black-box training poisoning, where the attacker has no knowledge of the model, is considered in [5], and also in [6].”