Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

By Kang Liu, Brendan Dolan-Gavitt, Siddharth Garg. 2018

1,338 citations recorded by the source corpus.

Deep neural networks (DNNs) provide excellent performance across a wide range of classification tasks, but their training requires high computational resources and is often outsourced to third parties. Recent work has shown that outsourced training introduces the risk that a malicious trainer will return a backdoored DNN that behaves normally on most inputs but causes targeted misclassifications or degrades the accuracy of the network when a trigger known only to the attacker is present. In this paper, we provide the first effective defenses against backdoor attacks on DNNs. We implement three backdoor attacks from prior work and use them to investigate two promising defenses, pruning and fine-tuning. We show that neither, by itself, is sufficient to defend against sophisticated attackers. We then evaluate fine-pruning, a combination of pruning and fine-tuning, and show that it successfully weakens or even eliminates the backdoors, i.e., in some cases reducing the attack success rate to 0% with only a \(0.4\%\) drop in accuracy for clean (non-triggering) inputs. Our work provides the first step toward defenses against backdoor attacks in deep neural networks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

How To Backdoor Federated Learning (2018, methodology): “Defenses against backdoors use techniques such as fine-pruning [40], filtering [66], or various types of clustering [8, 65].”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, methodology): “Instead of filtering out data, they rely on removing activation units that are inactive on clean data [56, 57].”
Can You Really Backdoor Federated Learning? (2019, background): “Existing approaches against backdoor attacks (28; 20; 31; 13; 32; 26) either require a careful examination of the training data or full control of the training process at the server, which may not apply in the federated learning case.”
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks (2019, background): “Fine-Pruning [15] removes backdoors by pruning redundant neurons less useful for normal classification.”
Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering (2018, background): “In Liu et al. (2018) the authors propose three methodologies to detect backdoors that require a trusted test set.”
Spectral Signatures in Backdoor Attacks (2018, methodology): “After the submission of this work, independent work by [23] proposes another approach to protection against backdoor attacks that relies on a certain type of neuron pruning, as well as re-training on clean data.”
Bypassing Backdoor Detection Algorithms in Deep Learning (2019, background): “Liu et al., 2018 [15] take a similar approach of pruning neurons based on the latent representations of a known clean set of inputs, operating under the assumption that the backdoor neurons are dormant for clean inputs.”
STRIP: a defence against trojan attacks on deep neural networks (2019, methodology): “Built upon the reversed trigger, there are several available techniques to mitigate or remove the trojan behavior, such as i) filtering the trojaned input, ii) patching the DNN model via pruning and iii) patching the DNN model via unlearning that can remove the trojan behavior from the trojaned DNN ”
Backdooring Convolutional Neural Networks via Targeted Weight Perturbations (2018, background): “Further, network fine-pruning [18] has been demonstrated to be an effective mechanism to remove other forms of backdoors in CNNs.”
Defending Neural Backdoors via Generative Distribution Modeling (2019, background): “However, it is not effective for pruning-aware backdoor attacks [16].”
PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning (2022, background): “For instance, a potentially poisoned classifier can be fine-tuned using a clean training dataset [37].”
Weight Poisoning Attacks on Pretrained Models (2020, background): “In conjunction with the poisoning literature, a variety of defense mechanisms have been developed, in particular pruning or further training of the poisoned model (Liu et al., 2017, 2018a), albeit sometimes at the cost of performance (Wang et al., 2019).”
Label-Consistent Backdoor Attacks (2019, methodology): “, 2019) or the latent representation of the resulting model (Tran et al., 2018; Liu et al., 2018; Chen et al., 2018).”
Poisoning the Unlabeled Dataset of Semi-Supervised Learning (2021, background): “Fine-tuning based defenses [34] take a (potentially poisoned) model and fine-tune it on clean, un-poisoned data.”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “In addition to FoolsGold that can defend against backdoor attacks, there are a number of detection algorithms that are designed to detect which inputs contain backdoor, and which parts of the model (its activation functions specifically) are responsible for triggering the adversarial behavior of the”
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks (2019, background): “There have also been very recent attempts at defending against backdoor attacks [36], [37]; however, defenses have only”
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, methodology): “In Figure 3, we show the attention maps of a backdoored WRN-16-1 after puriﬁed by four different mechanisms: Fine-pruning, the standard ﬁnetuning, MCR, and our NAD approach.”
Latent Backdoor Attacks on Deep Neural Networks (2019, methodology): “Thus Fine-Pruning is not a practical defense against latent backdoors.”