Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization

By Daniel Jakubovitz, Raja Giryes. 2018

243 citations recorded by the source corpus.

Deep neural networks have lately shown tremendous performance in various applications including vision and speech processing tasks. However, alongside their ability to perform these tasks with such high accuracy, it has been shown that they are highly susceptible to adversarial attacks: a small change of the input would cause the network to err with high confidence. This phenomenon exposes an inherent fault in these networks and their ability to generalize well. For this reason, providing robustness to adversarial attacks is an important challenge in networks training, which has led to an extensive research. In this work, we suggest a theoretically inspired novel approach to improve the networks' robustness. Our method applies regularization using the Frobenius norm of the Jacobian of the network, which is applied as post-processing, after regular training has finished. We demonstrate empirically that it leads to enhanced robustness results with a minimal change in the original network's accuracy.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Adversarial Training for Free! (2019, background): “, 2017], L2-nonexpansive nets [Qian and Wegman, 2018], Jacobian Regularization [Jakubovitz and Giryes, 2018], Universal Perturbation [Shafahi et al.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Generalization: Research also includes independent benchmarks of attacks and defenses [13, 15, 5, 6, 85], their properties [57, 84], as well as theoretical questions [34, 42, 23, 99, 27, 88, 102, 104].”
Learning Explainable Models Using Attribution Priors (2019, methodology): “This follows from prior literature on training neural networks to predict drug response [14].”
Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks (2018, unknown): “We hope future work explore more input-agnostic defences that minimize the sensitivity of large-scale models to small perturbations with techniques like model compression [20], Jacobian regularization [27, 30, 62, 69], or other types of denoising.”
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle (2019, methodology): “To improve the robustness of neural networks to adversarial examples, many defense strategies and models have been proposed, such as adversarial training [25], orthogonal regularization [6,22], L2 nonexpansive networks [28], Bayesian method [41], random masking [24], TRADES [42], rejecting adversari”
Improving performance of deep learning models with axiomatic attribution priors and expected gradients (2020, background): “Some recent attribution priors have proposed regularizing integrated gradients attributions [19, 5].”
On the Connection Between Adversarial Robustness and Saliency Map Interpretability (2019, result): “Similar identities were previously also independently derived in (Elsayed et al., 2018) and (Jakubovitz & Giryes, 2018).”
Learning Smooth Neural Functions via Lipschitz Regularization (2022, methodology): “Then we can encourage smoothness at these input samples by penalizing the norm of the Jacobian [Drucker and Le Cun 1991; Gulrajani et al. 2017; Hoffman et al. 2019; Jakubovitz and Giryes 2018; Varga et al. 2017] or the Hessian [Moosavi-Dezfooli et al. 2019].”
Robust Learning with Jacobian Regularization (2019, background): “8 Different incarnations of a similar idea have appeared in the following decades [Simard et al., 1992, Mitchell and Thrun, 1993, Aires et al., 1999, Rifai et al., 2011, Gulrajani et al., 2017, Yoshida and Miyato, 2017, Czarnecki et al., 2017, Jakubovitz and Giryes, 2018].”
You Only Propagate Once: Painless Adversarial Training Using Maximal Principle (2019, methodology): “To improve the robustness of neural networks to adversarial examples, many defense approaches have been proposed, like, adversarial trainining [5], orthogonal regularization [10,11], L2 nonexpansive networks [12], Bayesian method [13], random masking [14], TRADES [7], rejecting adversarial examples ”