Cascade Adversarial Machine Learning Regularized with a Unified Embedding

By Taesik Na, Jong Hwan Ko, Saibal Mukhopadhyay. 2017

107 citations recorded by the source corpus.

Injecting adversarial examples during training, known as adversarial training, can improve robustness against one-step attacks, but not for unknown iterative attacks. To address this challenge, we first show iteratively generated adversarial images easily transfer between networks trained with the same strategy. Inspired by this observation, we propose cascade adversarial training, which transfers the knowledge of the end results of adversarial training. We train a network from scratch by injecting iteratively generated adversarial images crafted from already defended networks in addition to one-step adversarial images from the network being trained. We also propose to utilize embedding space for both classification and low-level (pixel-level) similarity learning to ignore unknown pixel level perturbation. During training, we inject adversarial images without replacing their corresponding clean images and penalize the distance between the two embeddings (clean and adversarial). Experimental results show that cascade adversarial training together with our proposed low-level similarity learning efficiently enhances the robustness against iterative attacks, but at the expense of decreased robustness against one-step attacks. We show that combining those two techniques can also improve robustness under the worst case black box attack scenario.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples (2018, background): “Cascade adversarial machine learning (Na et al., 2018) is closely related to the above defense.”
Anti-Retroactive Interference for Lifelong Learning (2022, methodology): “The most common method is adversarial training [18,15,26] with adversarial examples added to the training data.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, methodology): “[85] employed a network that is regularized with a unified embedding for classification and low-level similarity learning.”
Adversarial Training Versus Weight Decay (2018, background): “Na et al. (2017) and Kannan et al. (2018) propose to add a term to the loss to encourage natural images and their adversarial counterparts to have similar logit representations.”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, background): “before fitting the model, or adaptively in the sense of adversarial training, with different augmented examples per training image generated in every iteration [21, 31, 48].”
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks (2019, background): “, 2018), cascade adversarial training (CASADV) (Na et al., 2018), randomization (Xie et al.”
Security and Privacy Issues in Deep Learning (2018, methodology): “Unified Embedding adversarial learning [Na et al. 2017]”
Adversarially Robust Generalization Just Requires More Unlabeled Data (2019, methodology): “We compare our proposed method with several baselines which use labeled training data only, including the original PGD-based adversarial training [24], thermometer encoding [5], cascade learning [29] and ADV-BNN [22].”