Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

By Xuanqing Liu, Yao Li, Chongruo Wu, Cho-Jui Hsieh. 2018

187 citations recorded by the source corpus.

We present a new algorithm to train a robust neural network against adversarial attacks. Our algorithm is motivated by the following two ideas. First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness. Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way. Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net. Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks. On CIFAR-10 with VGG network, our model leads to 14\% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu 2017) under PGD attack with $0.035$ distortion, and the gap becomes even larger on a subset of ImageNet.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Zhu
2. Hang Su
3. Matthias Hein
4. Tianyu Pang
5. Ajmal Saeed Mian
6. Bernt Schiele
7. Yinpeng Dong
8. Mubarak Shah

Citations in the corpus

• Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack (2018, methodology): “Last but not least, we also notice that recent work AdvBNN [28] also combines the adversarial training and noise injection on weight (i.”
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks (2020, background): “Empirical evidence supporting this intuition has also been presented (Liu et al., 2019; Wu et al., 2019, etc.).”
• Adversarial Robustness vs. Model Compression, or Both? (2019, background): “Since that researchers have begun to notice the issue when designing new defenses, more defenses have been proposed including adversarial training based ones [28, 37, 41] and others [39, 46].”
• Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “Similarly, a Bayesian neural network is proposed in [368] for adversarial robustness.”
• Benchmarking Adversarial Robustness on Image Classification (2020, background): “The randomness can also be modeled by Bayesian neural networks [33].”
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “[374] proposed a novel technique to train a robust neural network against adversarial attacks.”
• Adversarial Attacks and Defenses in Deep Learning (2020, unknown): “[99] combine the Bayesian neural network (BNN) [100] with adversarial training to learn the optimal model-weight distribution under adversarial attacks.”
• Relating Adversarially Robust Generalization to Flat Minima (2021, unknown): “Originally proposed in different variants in [56, 40, 24], it received considerable attention in [37, 16] and has been extended in various ways: [33, 7, 2] utilize interpolated or unlabeled examples, [57, 38] achieve robustness against multiple threat models, [54, 32, 64] augment AT with a reject op”
• Adversarially Robust Generalization Just Requires More Unlabeled Data (2019, methodology): “We compare our proposed method with several baselines which use labeled training data only, including the original PGD-based adversarial training [24], thermometer encoding [5], cascade learning [29] and ADV-BNN [22].”
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks (2019, methodology): “We consider 13 defenses recently developed: adversarial training (A DV - TRAIN ) (Madry et al., 2018), adversarial training of Bayesian DNNs (A DV -BNN) (Liu et al., 2019), Thermometer encoding (T HERM ) (Buckman et al., 2018), T HERM -”