Limitations of the Lipschitz constant as a defense against adversarial examples
By Todd P. Huster, C. Chiang, Ritu Chadha. 2018
90 citations recorded by the source corpus.
Several recent papers have discussed utilizing Lipschitz constants to limit the susceptibility of neural networks to adversarial examples. We analyze recently proposed methods for computing the Lipschitz constant. We show that the Lipschitz constant may indeed enable adversarially robust neural networks. However, the methods currently employed for computing it suffer from theoretical and practical limitations. We argue that addressing this shortcoming is a promising direction for future research into certified adversarial defenses.
Subfields: Artificial Intelligence, Molecular Biology, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- J. Kolter
- Ruslan Salakhutdinov
- Anima Anandkumar
- Huan Zhang
- Sergey Levine
- Roger Baker Grosse
- Matt Fredrikson
- Percy Liang
In Molecular Biology
- Erik Poll
- Joost Visser
- J. Kolter
- Ruslan Salakhutdinov
- Roger Baker Grosse
- Anima Anandkumar
- Matt Fredrikson
- Klas Leino
In Signal Processing
- Roger Baker Grosse
- Ruslan Salakhutdinov
- Kamalika Chaudhuri
- Anima Anandkumar
- Erik Poll
- Joost Visser
- A. Serban
- Saerom Park
Citations in the corpus
- Sorting out Lipschitz function approximation (2018, background): “Huster et al. [26] explored universal approximation properties of Lipschitz networks and proved that ReLU activations cannot be used to approximate the absolute value function with ∞ -norm constraints.”
- Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds (2021, background): “It leads to considerably lower clean accuracy in certified training compared to standard and adversarial training [5, 6].”
- A Closer Look at Accuracy vs. Robustness (2020, methodology): “[22] provide methods for imposing global Lipschitzness constraints; however, the state-of-the-art methods for training such networks do not lead to highly expressible functions.”
- Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks (2019, unknown): “, the Lipschitz constant [17,27,33,38].”
- Globally-Robust Neural Networks (2021, methodology): “Furthermore, as raised by Huster et al. (2018), an additional potential difficulty in using the global bound for certification is the estimation of the global bound itself.”
- Lipschitz-Certifiable Training with a Tight Outer Bound (2020, background): “However, the global Lipschitz constant can still overestimate the outer bound and impose a strong penalty for it, limiting the expressiveness of the model [17].”
- Adversarial Examples on Object Recognition (2020, methodology): “Moreover, Huster et al. [84] showed that the existing approaches to compute the Lipschitz constant of DNNs have representational learning limitations, which may limit the robustness guarantees we can obtain using it.”