Certifying Some Distributional Robustness with Principled Adversarial Training

By Aman Sinha, Hongseok Namkoong, John C. Duchi. 2017

971 citations recorded by the source corpus.

Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms. We address this problem through the principled lens of distributionally robust optimization, which guarantees performance under adversarial input perturbations. By considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data. For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization. Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss. For imperceptible perturbations, our method matches or outperforms heuristic approaches.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, methodology): “[79] Sinha, A.”
Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “, log loss), whereas PixelDP can certify each specific prediction, and also provides intuitive metrics like robust accuracy, which is not supported by [53].”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “…techniques from causal inference (Schölkopf, 2019), uncertainty estimation (Gal and Ghahramani, 2016; Kendall and Gal, 2017), density estimation and generative modeling (Kingma et al., 2014), distributional robustness (Sinha et al., 2017; Sagawa et al., 2019) and invariance (Arjovsky et al., 2019).”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “This immediately implies that certifying backdoor robustness is at least as hard as certifying robustness against adversarial samples [74].”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, methodology): “We consider the l ∞ -ball adversarial constraint throughout the paper, as it is widely adopted by most adversarial defense methods [16, 33, 34, 40, 50, 61, 66].”
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations (2019, methodology): “While we both use the Wasserstein distance in the context of adversarial training, the approach is quite different: Sinha et al. (2018) use the Wasserstein distance to perturb the underlying data distribution , whereas we use the Wasserstein distance as an attack model for perturbing each example .”
Stronger data poisoning attacks break data sanitization defenses (2018, background): “Designing models that are robust to such attacks, as well as coming up with more effective attacks, is an active area of research (Papernot et al., 2016c; Madry et al., 2017; Tramèr et al., 2017; Wong and Kolter, 2018; Raghunathan et al., 2018; Sinha et al., 2018; Athalye et al., 2018; Papernot and ”
Ensemble Adversarial Training: Attacks and Defenses (2017, background): “, `∞ bounded perturbations (Madry et al., 2017; Sinha et al., 2018; Raghunathan et al., 2018; Kolter & Wong, 2017)), yet they also fail to extend to other adversaries not considered at training time (see Appendix C.”
Adversarial Vulnerability of Neural Networks Increases With Input Dimension (2018, background): “Like for Goodfellow et al. (2015); Lyu et al. (2015); Sinha et al. (2018) and many others, a classifier ϕ will hence be robust if, on average over x, a small adversarial perturbation δ of x creates only a small variation δL of the loss.”
Invariant Risk Minimization (2019, background): “Second, we could minimize R(f) = maxe∈Etr R (f) − re, a robust learning objective where the constants re serve as environment baselines [2, 6, 15, 46].”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, methodology): “Complementary to this, others have developed methods based on distributional robustness for training models to be minimax optimal to perturbations of bounded magnitude in order to guard against adversarial attacks (Sinha et al., 2018) and improve generalization (Rothenh¨ausler et al., 2018).”
Conditional variance penalties and domain shift robustness (2017, background): “A Wasserstein-ball is a suitable class of distributions for example in the context of adversarial examples (Sinha et al., 2018; Szegedy et al., 2014; Goodfellow et al., 2015).”
Robustness May Be at Odds with Accuracy (2018, background): “Adversarial training The most successful approach to building adversarially robust models so far (Madry et al., 2017; Kolter & Wong, 2017; Sinha et al., 2017; Raghunathan et al., 2018) was so-called adversarial training (Goodfellow et al., 2014b).”
Adversarially Robust Generalization Requires More Data (2018, background): “As mentioned in the introduction, simpler datasets such as MNIST have recently seen significant progress in terms of `∞-robustness.”