Understanding adversarial training: Increasing local stability of supervised models through robust optimization

By Uri Shaham, Yutaro Yamada, Sahand N. Negahban. 2015

104 citations recorded by the source corpus.

Abstract We show that adversarial training of supervised learning models is in fact a robust optimization procedure. To do this, we establish a general framework for increasing local stability of supervised learning models using robust optimization. The framework is general and broadly applicable to differentiable non-parametric models, e.g., Artificial Neural Networks (ANNs). Using an alternating minimization-maximization procedure, the loss of the model is minimized with respect to perturbed examples that are generated at each parameter update, rather than with respect to the original training data. Our proposed framework generalizes adversarial training, as well as previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the networks also on the original test data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Evaluating the Robustness of Neural Networks (2016, background): “Given these threats, there have been various attempts [15], [2], [20], [42], [39] at constructing defenses that increase the robustness of a neural network, defined as a measure of how easy it is to find adversarial examples that are close to their original input.”
Adversarial Examples on Object Recognition (2020, unknown): “ [152] Design – Adversarial Training * * * * Gradient Regularization [114] Design – Adversarial Training * * * * Structured Regularization [139] Design – Adversarial Training * * ** * Robust Training [149] Design – Adversarial Training ** * ** ** Strong Adversary Training [79] Design – Adversarial T”
Adversarially Robust Distillation (2019, methodology): “Adversarial training is another method for encouraging robustness to adversarial attacks during training [24].”