Bypassing Backdoor Detection Algorithms in Deep Learning

By T. Tan, Reza Shokri. 2019

168 citations recorded by the source corpus.

Deep learning models are vulnerable to various adversarial manipulations of their training data, parameters, and input sample. In particular, an adversary can modify the training data and model parameters to embed backdoors into the model, so the model behaves according to the adversary's objective if the input contains the backdoor features, referred to as the backdoor trigger (e.g., a stamp on an image). The poisoned model's behavior on clean data, however, remains unchanged. Many detection algorithms are designed to detect backdoors on input samples or model parameters, through the statistical difference between the latent representations of adversarial and clean input samples in the poisoned model. In this paper, we design an adversarial backdoor embedding algorithm that can bypass the existing detection algorithms including the state-of-the-art techniques. We design an adaptive adversarial training algorithm that optimizes the original loss function of the model, and also maximizes the indistinguishability of the hidden representations of poisoned data and clean data. This work calls for designing adversary-aware defense mechanisms for backdoor detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

How To Backdoor Federated Learning (2018, background): “Furthermore, none of these defenses are effective even in the setting for which they were designed because they can be evaded by a defense-aware attacker [2, 64].”
Blind Backdoors in Deep Learning Models (2020, background): “In this case, evasion loss enforces the similarity of representations between backdoored and normal inputs [84].”
On the security relevance of weights in deep learning. (2020, background): “Due to the flexibility of the models, these attacks are currently limited to the misclassification of individual points [41] or the introduction of backdoors [24, 33, 36].”
Invisible Backdoor Attack with Sample-Specific Triggers (2020, result): “Currently, there was also some other work in this field [37, 22, 27].”
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks (2021, background): “To further improve the stealthiness against human visual inspection, techniques from adversarial example generation have also been adopted in crafting poisoned images [47, 26].”
Towards Probabilistic Verification of Machine Unlearning (2020, background): “However, these two methods suffer from adaptive attacks [26].”
Weight Poisoning Attacks on Pretrained Models (2020, background): “Furthermore, as evidenced in Tan and Shokri (2019) and our own work, such defenses are not foolproof.”
Backdoor Learning: A Survey (2020, background): “proposed and reached decent performance against previous attacks, almost all of them were bypassed by following stronger adaptive attacks [96], [97].”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “These backdoor detection algorithms can however be bypassed by maximizing the latent indistinguishability of backdoorenabled adversarial inputs and clean inputs [136].”