Hidden Trigger Backdoor Attacks

By Aniruddha Saha, Akshayvarun Subramanya, Hamed Pirsiavash. AAAI Conference on Artificial Intelligence, 2019

775 citations recorded by the source corpus.

With the success of deep learning algorithms in various domains, studying adversarial attacks to secure deep models in real world applications has become an important research topic. Backdoor attacks are a form of adversarial attacks on deep networks where the attacker provides poisoned data to the victim to train the model with, and then activates the attack by showing a specific small trigger pattern at the test time. Most state-of-the-art backdoor attacks either provide mislabeled poisoning data that is possible to identify by visual inspection, reveal the trigger in the poisoned data, or use noise to hide the trigger. We propose a novel form of backdoor attack where poisoned data look natural with correct labels and also more importantly, the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time. We perform an extensive study on various image classification settings and show that our attack can fool the model by pasting the trigger at random locations on unseen images although the model performs well on clean data. We also show that our proposed attack cannot be easily defended using a state-of-the-art defense algorithm for backdoor attacks.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/31675c0dea68199af03a8c58b638b89df4b9db3f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, methodology): “The Hidden Trigger Backdoor Attack [104] adapts the feature collision framework [33] to construct backdoor poison samples.”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “[39] propose to transform the backdoored images into benign-looking ones, which makes them harder to detect.”
Invisible Backdoor Attack with Sample-Specific Triggers (2020, methodology): “[34] assumed that attackers know model structure.”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “, preserve the semantics) and can even be invisible [32], triggers for text data are highly likely to introduce undesirably large change in the semantics.”
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021, methodology): “(Liu et al., 2018), ﬁnding methods to poison training images in a more concealed way (Saha et al., 2020; Liu et al., 2020) and generating dynamic triggers varying from input to input to escape from detection (Nguyen and Tran, 2020).”
Backdoor Learning: A Survey (2020, background): “proposed and reached decent performance against previous attacks, almost all of them were bypassed by following stronger adaptive attacks [96], [97].”
Customizing Triggers with Concealed Data Poisoning (2020, neutral): “…Work on Data Poisoning Most past work on data poisoning for neural models focuses on computer vision and looks to cause errors on speciﬁc examples (Shafahi et al., 2018; Koh and Liang, 2017) or when unnatural universal patches are present (Saha et al., 2020; Turner et al., 2018; Chen et al., 2017).”
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021, methodology): “Various backdoor attack methods are developed, and most of them are based on training data poisoning (Chen et al., 2017; Liao et al., 2018; Saha et al., 2020; Liu et al., 2020; Zhao et al., 2020).”
Blind Backdoors in Deep Learning Models (2020, background): “Another variant adds correctly labeled training inputs with backdoor patterns [70, 76, 92].”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, methodology): “An adversary can control the behaviors of a welltrained model as long as the model is trained on a poisoned dataset [22, 108].”
Anti-Backdoor Learning: Training Clean Models on Poisoned Data (2021, background): “These can be simple patterns such as a single pixel [6] and a black-white checkerboard [1], or more complex patterns such as blending backgrounds [11], natural reﬂections [12], invisible noise [13–16], adversarial patterns [17] and sample-wise patterns [18, 19].”
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks (2021, methodology): “The trigger is either built on a perturbation on the clean image [40] or warping-based [36] to activate the backdoor.”
Safe Distillation Box (2021, background): “For example, Saha, Subramanya, and Pirsiavash (2020) propose a novel form of backdoor attack where poisoned data look natural with correct labels and the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time.”
Towards Probabilistic Verification of Machine Unlearning (2020, background): “Our veri-ﬁcation mechanism can be easily extended to other backdoor attack methods [14, 23, 24].”
Concealed Data Poisoning Attacks on NLP Models (2021, background): “…Work on Data Poisoning Most past work on data poisoning for neural models focuses on computer vision and looks to cause errors on speciﬁc examples (Shafahi et al., 2018; Koh and Liang, 2017) or when unnatural universal patches are present (Saha et al., 2020; Turner et al., 2018; Chen et al., 2017).”
Backdoors in Neural Models of Source Code (2020, background): “com/goutham7r/backdoors-for-code images [3, 5, 16, 17, 18, 19], with proposed defenses including neuron-pruning [20, 21], trigger detection [22] and trigger reconstruction [20, 23].”
Explainable Deep Learning: A Field Guide for the Uninitiated (2020, background): “The vulnerability of a DNN to adversarial examples (Yuan et al., 2019; Goodfellow et al., 2014; Carlini & Wagner, 2017; Madry et al., 2017) and poisoned training sets (Saha et al., 2020) raises much concern on trustworthiness.”