Lipschitz Networks and Distributional Robustness

By Zac Cranko, Simon Kornblith, Zhan Shi, Richard Nock. 2018

12 citations recorded by the source corpus.

Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation. We bound the distributionally robust risk for a model class rich enough to include deep neural networks by a regularised empirical risk involving the Lipschitz constant of the model. This allows us to interpretand quantify the robustness properties of a deep neural network. As an application we show the distributionally robust risk upperbounds the adversarial training risk.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jean-Baptiste Cordonnier
2. Yihe Dong
3. Andreas Loukas
4. Sebastian U. Stich
5. Petar Velickovic
6. Martin Jaggi
7. Muhan Zhang
8. Xiyuan Wang

In Molecular Biology

1. Andreas Loukas
2. Jean-Baptiste Cordonnier

Citations in the corpus

• Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth (2021, background): “However, while increasing the Lipschitz constants slows down residual convergence, it carries the side effect of making the MLPs more sensitive to input perturbations, and thus is often associated with less robust models (Cranko et al., 2018).”