Spectral Signatures in Backdoor Attacks

By Brandon Tran, Jerry Zheng Li, Aleksander Madry. 2018

1,009 citations recorded by the source corpus.

A recent line of work has uncovered a new form of data poisoning: so-called \emph{backdoor} attacks. These attacks are particularly dangerous because they do not affect a network's behavior on typical, benign data. Rather, the network only deviates from its expected output when triggered by a perturbation planted by an adversary. In this paper, we identify a new property of all known backdoor attacks, which we call \emph{spectral signatures}. This property allows us to utilize tools from robust statistics to thwart the attacks. We demonstrate the efficacy of these signatures in detecting and removing poisoned examples on real image sets and state of the art neural network architectures. We believe that understanding spectral signatures is a crucial first step towards designing ML systems secure against such backdoor attacks

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “making them distinguishable from clean data [59].”
Advances and Open Problems in Federated Learning (2019, methodology): “r use untargeted attacks to 67 degrade the effectiveness of adversarial training. One possible defense against combined training- and inference-time adversaries are methods to detect backdoor attacks [387, 96, 398, 107]. Difﬁculties in applying previous defenses (such as those cited above) to the fe”
How To Backdoor Federated Learning (2018, methodology): “Defenses against backdoors use techniques such as fine-pruning [40], filtering [66], or various types of clustering [8, 65].”
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, background): “One category of defenses [4, 15, 56, 59] detects malicious data based on their (negative) impact on the performance of the learnt model.”
Backdoor Attacks on Federated Meta-Learning (2020, methodology): “In contrast with other defense mechanisms for federated learning [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], our method does not require any third-party to examine user updates, and it is thus compatible with secure update aggregation methods.”
Can You Really Backdoor Federated Learning? (2019, background): “Existing approaches against backdoor attacks (28; 20; 31; 13; 32; 26) either require a careful examination of the training data or full control of the training process at the server, which may not apply in the federated learning case.”
A Little Is Enough: Circumventing Defenses For Distributed Learning (2019, background): “A few defenses aimed at detecting backdoors were proposed [26, 22, 6, 27], but those defenses assume a single-server training in which the backdoor is injected in the training set for which the server has access to, so that by clustering or other techniques the backdoor samples can be found and remo”
Bypassing Backdoor Detection Algorithms in Deep Learning (2019, methodology): “Figure 7: Correlation of latent representations of all inputs in the training dataset with the top eigenvector for the data filtering defense based on spectral signatures [25].”
STRIP: a defence against trojan attacks on deep neural networks (2019, background): “So if the defender is allowed to have a set of trojaned inputs as assumed in [20], [21], our STRIP appears to be able to detect class-specific trojan attacks; by carefully examining and analysing the entropy distribution of tested samples (done offline) because the entropy distribution of trojaned i”
Anti-Backdoor Learning: Training Clean Models on Poisoned Data (2021, methodology): “In this section, we compare the isolation precision of our ABL with two backdoor detection methods, namely Activation Clustering (AC) [7] and Spectral Signature Analysis (SSA) [6].”
Learning with Bad Training Data via Iterative Trimmed Loss Minimization (2018, methodology): “We inject backdoor images using exactly the same process as described in (Tran et al., 2018), i.”
Blind Backdoors in Deep Learning Models (2020, methodology): “, [9, 50, 91], are easily evaded in a similar way.”
Hidden Trigger Backdoor Attacks (2019, methodology): “We used the default 85% percentile threshold in (Tran, Li, and Madry 2018) which should ﬁnd 135 poisoned images out of 800 images where there are only 100 actual poisoned images.”
Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection (2019, background): “[40] propose another defense against backdoor attack, based on Spectral Signatures (SS) of representations.”
TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems (2019, methodology): “e corresponding network (e.g., [28]) as well as (3) restoring a trigger from an infected model and patch the model with that trigger (i.e., [48]). For the first method, the state-of-the-art technique [47] utilizes a new statistical method to examine training data and thus tracks down the training da”
Radioactive data: tracing through training (2020, positive): “Similarly the “backdoor” techniques are easy to identify and circumvent with outlier detection (Tran et al., 2018).”
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks (2021, background): “Several categories of defensive mechanisms have been developed in recent literature to counter backdoor attacks, including detection [8, 48, 18, 45], input mitigation [32, 27], model mitigation [28, 11, 50, 9, 38] approaches.”
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks (2019, background): “Another very recent approach [38] does not require a user to re-train the model, but assumes that the user has access to both clean and backdoored inputs, which is not the case in our attack scenario.”
Invisible Backdoor Attack with Sample-Specific Triggers (2020, methodology): “For Spectral Signatures, we report the outlier score for each sample, where a larger score denotes the sample is more likely poisoned.”