Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy

By Min Du, Ruoxi Jia, Dawn Song. 2019

201 citations recorded by the source corpus.

Outlier detection and novelty detection are two important topics for anomaly detection. Suppose the majority of a dataset are drawn from a certain distribution, outlier detection and novelty detection both aim to detect data samples that do not fit the distribution. Outliers refer to data samples within this dataset, while novelties refer to new samples. In the meantime, backdoor poisoning attacks for machine learning models are achieved through injecting poisoning samples into the training dataset, which could be regarded as "outliers" that are intentionally added by attackers. Differential privacy has been proposed to avoid leaking any individual's information, when aggregated analysis is performed on a given dataset. It is typically achieved by adding random noise, either directly to the input dataset, or to intermediate results of the aggregation mechanism. In this paper, we demonstrate that applying differential privacy can improve the utility of outlier detection and novelty detection, with an extension to detect poisoning samples in backdoor attacks. We first present a theoretical analysis on how differential privacy helps with the detection, and then conduct extensive experiments to validate the effectiveness of differential privacy in improving outlier detection, novelty detection, and backdoor attack detection.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “tently change the model’s output [36, 65], (2) use anomalies in the model’s internal behavior to identify poisoned inputs in the training data [12, 15, 59], or (3) prevent rare features in the training data from influencing the model [19, 28, 35].”
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2020, background): “Meanwhile, a large body of studies propose different approaches to defend DNN models against backdoor attacks (Liu et al., 2017, 2018a; Qiao et al., 2019; Du et al., 2020).”
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021, background): “On the other hand, a large body of research has proposed diverse defenses against backdoor attacks for images (Liu et al., 2018a; Wang et al., 2019; Qiao et al., 2019; Kolouri et al., 2020; Du et al., 2020).”
Backdoor Learning: A Survey (2020, methodology): “Natural Image Recognition MNIST [83] 28× 28 60,000 10,000 [46], [8], [59], [60], [48] [52], [65], [7], [66], [67] [15], [35], [39], [45], [56] [61], [62], [73], [74], [84] [69], [72] Fashion MNIST [85] 28× 28 60,000 10,000 [63], [69]”
Blind Backdoors in Deep Learning Models (2020, background): “Suppressing outliers Gradient shaping [32], DPSGD [18]”
FLAME: Taming Backdoors in Federated Learning (2022, background): “, [17]) by providing theoretical proof for linear models and validating it for DNNs empirically.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “Among other specialized defenses, differential privacy is used in [434] to detect poisoning samples for backdoor attacks.”