Defending Neural Backdoors via Generative Distribution Modeling

By Ximing Qiao, Yukun Yang, Hai Helen Li. 2019

205 citations recorded by the source corpus.

Neural backdoor attack is emerging as a severe security threat to deep learning, while the capability of existing defense methods is limited, especially for complex backdoor triggers. In the work, we explore the space formed by the pixel values of all possible backdoor triggers. An original trigger used by an attacker to build the backdoored model represents only a point in the space. It then will be generalized into a distribution of valid triggers, all of which can influence the backdoored model. Thus, previous methods that model only one point of the trigger distribution is not sufficient. Getting the entire trigger distribution, e.g., via generative modeling, is a key of effective defense. However, existing generative modeling techniques for image generation are not applicable to the backdoor scenario as the trigger distribution is completely unknown. In this work, we propose max-entropy staircase approximator (MESA) for high-dimensional sampling-free generative modeling and use it to recover the trigger distribution. We also develop a defense technique to remove the triggers from the backdoored model. Our experiments on Cifar10/100 dataset demonstrate the effectiveness of MESA in modeling the trigger distribution and the robustness of the proposed defense method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xingjun Ma
2. Bo Li
3. Dawn Song
4. Yisen Wang
5. Zhiyuan Liu
6. Maosong Sun
7. Tom Goldstein
8. Lingjuan Lyu

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Tom Goldstein
3. Bo Li
4. Baoyuan Wu
5. Xiaogang Wang
6. Alan L. Yuille
7. Dacheng Tao
8. Xingjun Ma

In Signal Processing

1. Vitaly Shmatikov
2. Bo Li
3. Xingjun Ma
4. Aleksander Madry
5. Dimitris Tsipras
6. Eugene Bagdasarian
7. Tom Goldstein
8. Baoyuan Wu

Citations in the corpus

• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “structing a single trigger, the authors of [140], [150], and 1572 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.”
• LIRA: Learnable, Imperceptible and Robust Backdoor Attacks (2021, background): “Several categories of defensive mechanisms have been developed in recent literature to counter backdoor attacks, including detection [8, 48, 18, 45], input mitigation [32, 27], model mitigation [28, 11, 50, 9, 38] approaches.”
• Invisible Backdoor Attack with Sample-Specific Triggers (2020, result): “Similar ideas were also studied [30, 9, 42], where they adopted different approaches for”
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, methodology): “These “remedy data” can subsequently be used in retraining to alleviate the impact of backdoor triggers (Wang et al., 2019a; Qiao et al., 2019).”
• Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021, background): “On the other hand, a large body of research has proposed diverse defenses against backdoor attacks for images (Liu et al., 2018a; Wang et al., 2019; Qiao et al., 2019; Kolouri et al., 2020; Du et al., 2020).”
• Adversarial Neuron Pruning Purifies Backdoored Deep Models (2021, methodology): “To repair the risk model, one direct way is to first reconstruct an approximation of the backdoor trigger via adversarial perturbation [43] or generative adversarial network (GAN) [5, 33, 50].”
• Blind Backdoors in Deep Learning Models (2020, methodology): “Input perturbation NeuralCleanse [95], ABS [54], TABOR [30], STRIP [24], Neo [93], MESA [69], Titration analysis [21]”
• ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2020, background): “Meanwhile, a large body of studies propose different approaches to defend DNN models against backdoor attacks (Liu et al., 2017, 2018a; Qiao et al., 2019; Du et al., 2020).”
• Backdoor Learning: A Survey (2020, background): “Trigger Synthesis based Defenses Adopt third-party model [52], [53], [54], [55], [56], [57] [58] Model Diagnosis based Defenses Adopt third-party model [59], [60], [61] Poison Suppression based Defenses Adopt third-party dataset [62], [63]”