Adversarial Weight Perturbation Helps Robust Generalization

By Dongxian Wu, Shutao Xia, Yisen Wang. 2020

894 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Unsolved Problems in ML Safety (2021, background): “Others could improve adversarial training methods [201] and find better losses [206, 179].”
Continual Learning by Modeling Intra-Class Variation (2022, background): “Despite focusing on different applications, WAP has intrinsic connections to (Wu et al., 2020; Zheng et al., 2021) where adversarial weight perturbation is shown to converge to flat minima (Hochreiter & Schmidhuber, 1997) and is beneficial to generalization.”
Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning (2021, methodology): “The overfitting problem of DNNs is often attributed to the complex loss landscape containing multiple local optima, and the sharpness of the loss landscape has been widely used to characterize the generalization gap in standard training scenarios from both theoretical and empirical perspectives [27,”
Backdoor Learning: A Survey (2020, background): “One well-studied example is adversarial examples [1], [2], [3], [4], [5], [6], which explored the adversarial vulnerability of DNNs at the inference stage.”
Diametrical Risk Minimization: theory and computations (2019, positive): “Concurrent to the present work, [45] develops an approach based on perturbation of parameters and data with strong empirical performance.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[360] showed that many adversarial training improvements appearing in the literature, e.”
Adversarial Robustness with Semi-Infinite Constrained Learning (2021, background): “For this reason, state-of-the-art adversarial attacks increasingly rely on heuristics such as random initializations, multiple restarts, pruning, and other ad hoc training procedures [40–49].”
Relating Adversarially Robust Generalization to Flat Minima (2021, methodology): “In our experiments, we consider several popular variants [62, 60, 72, 7, 22].”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, background): “B C OMPARISON WITH CONCURRENT WORKS Independently of our work, Pang et al. (2020a) and Wu et al. (2020) also investigate the limits of current approaches to adversarial training.”
Data Augmentation Can Improve Robustness (2021, methodology): “Notably, Carmon et al. [7], Hendrycks et al. [25], Najaﬁ et al. [35], Uesato et al. [53], Zhai et al. [59] showed that using additional data improves adversarial robustness, while Gowal et al. [20], Rice et al. [43], Wu et al. [55] found that data augmentation techniques did not boost robustness.”
Questions for Flat-Minima Optimization of Modern Neural Networks (2022, background): “Related Work (SAM) Concurently to Foret et al. (2021), Wu et al. (2020) propose an almost identical minimax ob-jective, additionally including perturbations of the training data, and layer-dependent scaling of ρ .”
Diffusion Models for Adversarial Purification (2022, neutral): “We can see that our method achieves both better (Wu et al., 2020) 88.25 62.11 (Gowal et al., 2020) 89.48 62.70 (Wu et al., 2020) 85.36 59.18 (Rebufﬁ et al., 2021) 87.33 61.72 (Gowal et al., 2021) 87.50 65.24 Ours 89.02 ± 0.21 70.64 ± 0.39 WideResNet-70-16 (Gowaletal.,2020) 91.10 66.02 (Rebufﬁ et…”
Fixing Data Augmentation to Improve Adversarial Robustness (2021, methodology): “• To the contrary of Rice et al. (2020); Wu et al. (2020); Gowal et al. (2020) which all tried data augmentation techniques without success, we are able to use any of these three aforementioned techniques to obtain new state-of-the-art robust accuracies.”
Cross-Entropy Loss Functions: Theoretical Analysis and Applications (2023, methodology): “(Wu et al., 2020) is significantly outperformed by ADV-COMP-SUM with a slightly larger model WRN-34-20, by more than 1.2% in the robust accuracy and also in the clean accuracy.”
Regularizing Neural Networks via Adversarial Model Perturbation (2020, methodology): “A similar technique designed by [46] suggests that flat minima also improve robust generalization.”
Do Wider Neural Networks Really Help Adversarial Robustness? (2020, background): “Very recently, [66] proposed to conduct adversarial weight perturbation aside from input perturbation to obtain more robust models.”