Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption

By Xu Sun, Zhiyuan Zhang, Xuancheng Ren, Ruixuan Luo, Liangyou Li. 2020

47 citations recorded by the source corpus.

We argue that the vulnerability of model parameters is of crucial value to the study of model robustness and generalization but little research has been devoted to understanding this matter. In this work, we propose an indicator to measure the robustness of neural network parameters by exploiting their vulnerability via parameter corruption. The proposed indicator describes the maximum loss variation in the non-trivial worst-case scenario under parameter corruption. For practical purposes, we give a gradient-based estimation, which is far more effective than random corruption trials that can hardly induce the worst accuracy degradation. Equipped with theoretical support and empirical validation, we are able to systematically investigate the robustness of different model parameters and reveal vulnerability of deep neural networks that has been rarely paid attention to before. Moreover, we can enhance the models accordingly with the proposed adversarial corruption-resistant training, which not only improves the parameter robustness but also translates into accuracy elevation.

Subfields: Artificial Intelligence, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Sharpness-Aware Minimization for Efficiently Improving Generalization (2020, background): “The concurrent work of Sun et al. (2020) focuses on resilience to random and adversarial corruption to expose a model’s vulnerabilities; this work is perhaps closest to ours.”
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021, background): “Inspired by studies that show flat minima tends to generalize better (Keskar et al., 2017; Sun et al., 2020; Foret et al., 2021), we can further prove CHILD-TUNINGF decreases the generalization error bound.”
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks (2021, background): “…generalization bounds, loss surface sharpness and minimization of the derived generalization bound have proven to be effective in attaining state-of-the-art performances in various tasks (Hochreiter & Schmidhuber, 1997; Mobahi, 2016; Chaudhari et al., 2019; Sun et al., 2020; Yue et al., 2020).”