Certified Robustness to Adversarial Examples with Differential Privacy

By Mathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel J. Hsu, Suman Sekhar Jana. IEEE Symposium on Security and Privacy, 2018

1,063 citations recorded by the source corpus.

Adversarial examples that fool machine learning models, particularly deep neural networks, have been a topic of intense research interest, with attacks and defenses being developed in a tight back-and-forth. Most past defenses are best effort and have been shown to be vulnerable to sophisticated attacks. Recently a set of certified defenses have been introduced, which provide guarantees of robustness to norm-bounded attacks. However these defenses either do not scale to large datasets or are limited in the types of models they can support. This paper presents the first certified defense that both scales to large networks and datasets (such as Google’s Inception network for ImageNet) and applies broadly to arbitrary model types. Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability, Privacy & Security

View Paper: https://arxiv.org/abs/1802.03471

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

In Privacy & Security

Citations in the corpus

Unsolved Problems in ML Safety (2021, background): “Researchers could improve adversarial robustness certifications [144, 116, 37], so that models have verifiable adversarial robustness.”
Privacy in Deep Learning: A Survey (2020, background): “Finding this sensitivity is not always trivial, specifically if the function f is a deep neural network, or even a number of layers of it [55].”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “ing data on the model by minimizing the loss over a lp ball of each training point, leading to more training data memorization. In addition, we find that a recently-proposed robust training algorithm [29], which adds a noise layer for robustness, also leads to an increase of membership inference acc”
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness (2019, background): “In fact, MNIST is the only dataset for which robustness to adversarial examples is considered even remotely close to being solved (Schott et al., 2019) and researchers working on (provable) robustness to adversarial examples have moved on to other, larger vision datasets such as CIFAR-10 (Madry et a”
Data Poisoning against Differentially-Private Learners: Attacks and Defenses (2019, background): “Previous work [13] proposed defenses based on differential privacy, but for test-time attacks.”
Certified Adversarial Robustness via Randomized Smoothing (2019, methodology): “Tables for ImageNet and CIFAR-10 are in Appendix D. Prior randomized smoothing bounds Figure 6a plots the certified accuracy obtained using our bound alongside the certified accuracy obtained using the bounds of Lecuyer et al. (2019) and Li et al. (2018).”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “Noise can be drawn either from Gaussian [26, 24, 37], Laplace [24], Uniform [44], or Multinomial [12] distributions.”
Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, methodology): “Beyond that, several recent works utilizing similar concept as ours in their defense method are discussed as well, including certified robustness [26], random self-ensemble [27], and Adv-BNN [28].”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “[18]) which also adopts very similar idea, except that they view this problem from the angle of differential privacy, while we believe that the adversarial robustness is more correlated with regularization and ensemble learning.”
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (2020, background): “not generalizing well to unseen data [66] [67] [68]).”
On Evaluating Adversarial Robustness (2019, background): “While this approach of designing-for-provability has seen excellent progress—the best approaches today can certify some (small) robustness even on ImageNet classifiers (Lecuyer et al., 2018)— often the best heuristic defenses offer orders of magnitude better (estimated) robustness.”
Unlabeled Data Improves Adversarial Robustness (2019, positive): “Improving on previous work [24, 26], Cohen et al.”
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers (2019, methodology): “[21] first proved robustness guarantees for randomized smoothing classifier, utilizing inequalities from the differential privacy literature.”
Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, background): “In addition, adversarial training [52] and differential privacy [53, 54] are proven effective to create a robust machine learning model [55].”
SplitFed: When Federated Learning Meets Split Learning (2020, methodology): “Then, stability bounds for the expected output of the DP function are given by the following Lemma (Lecuyer et al. 2019):”