Regularizing deep networks using efficient layerwise adversarial training

By S. Sankaranarayanan, Arpit Jain, Ramalingam Chellappa, Ser-Nam Lim. 2017

103 citations recorded by the source corpus.

Adversarial training has been shown to regularize deep neural networks in addition to increasing their robustness to adversarial examples. However, the regularization effect on very deep state of the art networks has not been fully investigated. In this paper, we present a novel approach to regularize deep neural networks by perturbing intermediate layer activations in an efficient manner. We use these perturbations to train very deep models such as ResNets and WideResNets and show improvement in performance across datasets of different sizes such as CIFAR-10, CIFAR-100 and ImageNet. Our ablative experiments show that the proposed approach not only provides stronger regularization compared to Dropout but also improves adversarial robustness comparable to traditional adversarial training approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Liu
2. Dilin Wang
3. Ajmal Saeed Mian
4. S. Sankaranarayanan
5. Ramalingam Chellappa
6. Naveed Akhtar
7. Carlos Domingo Castillo
8. Chengyue Gong

In Computer Vision and Pattern Recognition

1. Qiang Liu
2. Ramalingam Chellappa
3. Ajmal Saeed Mian
4. Carlos Domingo Castillo
5. S. Sankaranarayanan
6. Rajeev Ranjan
7. Dilin Wang
8. Luc Van Gool

In Radiology, Nuclear Medicine and Imaging

1. S. Sankaranarayanan
2. Qiang Liu
3. Ramalingam Chellappa

Citations in the corpus

• Improving Neural Language Modeling via Adversarial Training (2019, methodology): “More related to our work, (Sankaranarayanan et al., 2018) proposes a layer-wise adversarial training method to regularize deep neural networks.”
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “see [23], [90]) to reduce over-fitting, which in turn improves robustness of the networks against the adversarial attacks.”
• Improving Network Robustness against Adversarial Attacks with Compact Convolution (2017, background): “Several recent works have been proposed to improve robustness, such as adversarial training [2,6], gradient masking [7], etc.”