Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser

By Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Jun Zhu, Xiaolin Hu. 2017

1,036 citations recorded by the source corpus.

Neural networks are vulnerable to adversarial examples, which poses a threat to their application in security sensitive systems. We propose high-level representation guided denoiser (HGD) as a defense for image classification. Standard denoiser suffers from the error amplification effect, in which small residual adversarial noise is progressively amplified and leads to wrong classifications. HGD overcomes this problem by using a loss function defined as the difference between the target model's outputs activated by the clean image and denoised image. Compared with ensemble adversarial training which is the state-of-the-art defending method on large images, HGD has three advantages. First, with HGD as a defense, the target model is more robust to either white-box or black-box adversarial attacks. Second, HGD can be trained on a small subset of the images and generalizes well to other images and unseen classes. Third, HGD can be transferred to defend models other than the one guiding it. In NIPS competition on defense against adversarial attacks, our HGD solution won the first place and outperformed other models by a large margin.1

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Training for Free! (2019, methodology): “To put things further in perspective, Uesato et al. [2018] broke three defenses for (cid:15) = 2 non-targeted attacks on ImageNet [Guo et al., 2017, Liao et al., 2018, Xie et al., 2017], degrading their performance below 1%.”
Feature Denoising for Improving Adversarial Robustness (2018, methodology): “[15] propose to use highlevel features to guide the pixel denoiser; in contrast, our denoising is applied directly on features.”
Towards the first adversarially robust neural network model on MNIST (2018, methodology): “Other ideas in similar directions include the use of denoising autoencoders in (Gu & Rigazio, 2014) and (Liao et al., 2017) as well as MagNets (Meng & Chen, 2017) (which projects or rejects inputs depending on their distance to the data manifold).”
On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses (2018, methodology): “High-level representation Guided Denoiser (HGR) (Liao et al., 2018) proposes denoising inputs using a trained neural network before passing them to a standard classiﬁer.”
Adversarial Examples Are a Natural Consequence of Test Error in Noise (2019, background): “…2018; Das et al., 2018; 2017), Pixel Deﬂection (Prakash et al., 2018), total variance minimization (Guo et al., 2017), respresentation-guided denoising (Liao et al., 2018), and random resizing and random padding of the input image (Xie et al., 2017). failed to improve performance in Gaussian noise.”
Adversarial Attacks and Defences: A Survey (2018, methodology): “Liao et al [46] introduced HGD to devise a robust target model immune against white-box and black-box adversarial attacks.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “Defenses: Proposed defenses include detection and rejection methods [31, 25, 54, 60, 3, 63], pre-processing, quantization and dimensionality reduction methods [12, 73, 7], manifold-projection methods [39, 72, 82, 86], methods based on stochasticity/regularization or adapted architectures [109, 7, 68”
Diametrical Risk Minimization: theory and computations (2019, methodology): “Adversarial training [25] is a closely re-lated approach where the worst-case is computed by perturbing the data directly as for classical robust M-estimators in statistics; see also [21, 47, 10, 7] for other eﬀorts to achieve NNs that are robust against data perturbations.”
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (2019, background): “For example, when attacking HGD [32] with = 16, RHP outperforms FGSM [18] by 24.”
ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples (2018, background): “Similarly, [14] regards the imperceptible perturbations as the noises, and designs a high-level representation guided denoiser (HGD) to remove these noises.”
Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training (2019, background): “Many encouraging progresses been made towards improving model robustness against adversarial examples under different scenarios [58, 36, 33, 67, 72, 16, 71].”
A Survey of Black-Box Adversarial Attacks on Computer Vision Models (2019, methodology): “• Guided Denoiser: Since adversarial samples are based on adding noise to clean images, the natural idea that came to the authors was to develop denoising models as a defense mechanism [58].”
Robust Perception through Analysis by Synthesis (2018, methodology): “Other ideas in similar directions include the use of denoising autoencoders in [Gu and Rigazio, 2014] and [Liao et al., 2017], MagNets [Meng and Chen, 2017] (which reform adversarials to lie on the data manifold unless they are too far away in which case they are detected ).”
Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay (2024, methodology): “To ensure the reliability of DNNs in various scenarios, many defense methods have been proposed to maintain the robustness of DNN against adversarial attacks [16, 25, 34, 48, 54].”