Adversarial Neuron Pruning Purifies Backdoored Deep Models

By Dongxian Wu, Yisen Wang. Neural Information Processing Systems, 2021

410 citations recorded by the source corpus.

As deep neural networks (DNNs) are growing larger, their requirements for computational resources become huge, which makes outsourcing training more popular. Training in a third-party platform, however, may introduce potential risks that a malicious trainer will return backdoored DNNs, which behave normally on clean samples but output targeted misclassifications whenever a trigger appears at the test time. Without any knowledge of the trigger, it is difficult to distinguish or recover benign DNNs from backdoored ones. In this paper, we first identify an unexpected sensitivity of backdoored DNNs, that is, they are much easier to collapse and tend to predict the target label on clean samples when their neurons are adversarially perturbed. Based on these observations, we propose a novel model repairing method, termed Adversarial Neuron Pruning (ANP), which prunes some sensitive neurons to purify the injected backdoor. Experiments show, even with only an extremely small amount of clean data (e.g., 1%), ANP effectively removes the injected backdoor without causing obvious performance degradation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f17b9c4118e6cf6042e9fb9feacbea8909e5e655

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Backdoor Learning: A Survey (2020, positive): “A similar idea was further explored in [128], where they used adversarial weight perturbation (AWP) to amplify the differences between benign and malicious neurons.”
Poisoning Web-Scale Training Datasets is Practical (2023, positive): “…[113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this…”
Anti-Backdoor Learning: Training Clean Models on Poisoned Data (2021, methodology): “The current state-of-the-art erasing methods are Mode Connectivity Repair (MCR) [9], Neural Attention Distillation (NAD) [10] and Adversarial Neuron Pruning (ANP) [31].”