Robust Learning with Jacobian Regularization
By Judy Hoffman, Daniel A. Roberts, Sho Yaida. 2019
218 citations recorded by the source corpus.
Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks. The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yee Whye Teh
- Joost R. van Amersfoort
- Lewis Smith
- Sanja Fidler
- Yarin Gal
- Yoshua Bengio
- Ruslan Salakhutdinov
- Sergey Levine
Citations in the corpus
- Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, background): “This approach was previously discussed in the context of making neural networks more robust by Hoffman et al. (2019).”
- Learning Smooth Neural Functions via Lipschitz Regularization (2022, methodology): “Our Lipschitz regularization also leads to a smoother network compared to other weight decay measured by a popular metric, square Jacobian norm.”