Interpreting Adversarial Robustness: A View from Decision Surface in Input Space

By Fuxun Yu, Chenchen Liu, Yanzhi Wang, Liang Zhao, Xiang Chen. 2018

30 citations recorded by the source corpus.

One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing decision surfaces in both parameter space and input space, we instead show that the geometry property of decision surface in input space correlates well with the adversarial robustness. We then propose an adversarial robustness indicator, which can evaluate a neural network's intrinsic robustness property without testing its accuracy under adversarial attacks. Guided by it, we further propose our robust training method. Without involving adversarial training, our method could enhance network's intrinsic adversarial robustness against various adversarial attacks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Anqi Mao
2. Ali Shafahi
3. Mahyar Najibi
4. Tom Goldstein
5. Zheng Xu
6. Larry S. Davis
7. Dongxian Wu
8. Mehryar Mohri

Citations in the corpus

• Adversarial Training for Free! (2019, background): “Another popular defense strategy is to regularize the training loss using label smoothing, logit squeezing, or a Jacobian regularization [Shafahi et al., 2019a, Mosbach et al., 2018, Ross and Doshi-Velez, 2018, Hein and Andriushchenko, 2017, Jakubovitz and Giryes, 2018, Yu et al., 2018].”
• Adversarial Weight Perturbation Helps Robust Generalization (2020, methodology): “[59] tried to establish this connection in adversarial training but failed due to the inaccurate weight loss landscape characterization.”
• Cross-Entropy Loss Functions: Theoretical Analysis and Applications (2023, methodology): “, 2020), weight averaging/perturbation (Gowal et al., 2020; Wu et al., 2020; Tsai et al., 2021; Yu et al., 2018; Prabhu et al., 2019) and other techniques (Zhang & Wang, 2019; Qin et al.”