Metric Learning for Adversarial Robustness

By Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, Baishakhi Ray. 2019

204 citations recorded by the source corpus.

Deep networks are well-known to be fragile to adversarial attacks. We conduct an empirical analysis of deep representations under the state-of-the-art attack method called PGD, and find that the attack causes the internal representation to shift closer to the ``false'' class. Motivated by this observation, we propose to regularize the representation space under attack with metric learning to produce more robust classifiers. By carefully sampling examples for metric learning, our learned representation not only increases robustness, but also detects previously unseen adversarial samples. Quantitative experiments show improvement of robustness accuracy by up to 4% and detection efficiency by up to 6% according to Area Under Curve score over prior work. The code of our work is available at https://github.com/columbia/Metric_Learning_Adversarial_Robustness.

Subfields: Artificial Intelligence, Molecular Biology, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhe Gan
2. Linjie Li
3. Jingjing Liu
4. Yu Cheng
5. Yen-Chun Chen
6. Jianfeng Gao
7. Tianyu Pang
8. Hang Su

In Molecular Biology

1. Shuicheng Yan
2. Erik Poll
3. Zhe Gan
4. Joost Visser
5. Yu Cheng
6. Jun Zhu
7. Kun Xu
8. Tianyu Pang

In Representation & Self-Supervised Learning

1. Zhe Gan
2. Linjie Li
3. Jingjing Liu
4. Yu Cheng
5. Chen Zhu
6. Shuicheng Yan
7. Erik Poll
8. Chao Du

In Safety, Robustness & Interpretability

1. Hang Su
2. Jun Zhu
3. Yinpeng Dong
4. Tianyu Pang
5. Xiao Yang
6. Zhe Gan
7. Yu Cheng
8. Chen Zhu

Citations in the corpus

• Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, background): “Adversarial Training Many efforts have been devoted to improving AT from different angles: (i) use triplet-wise metric learning [8, 7] and optimal transport [20] to leverage inter-sample interactions; (ii) exploit extra unlabeled training data [12, 1]; and (iii) accelerate the training procedure [11”
• Deep Learning Based Vulnerability Detection: Are We There Yet? (2020, methodology): “To maximize the separation between the vulnerable and the neutral examples in the latent space, we adopt the triplet loss [42] as our loss function.”
• Boosting Adversarial Training with Hypersphere Embedding (2020, methodology): “Previous work finds that the adversarial attack would cause the internal representation to shift closer to the false class (Mao et al., 2019; Li et al., 2019).”
• Better Diffusion Models Further Improve Adversarial Training (2023, methodology): “…datasets or generated data, several enhancements for AT have been made employing strategies inspired by other areas, including metric learning (Mao et al., 2019; Pang et al., 2020a;b), self-supervised learning (Chen et al., 2020a;b; Naseer et al., 2020; Wang & Liu, 2022), ensemble learning…”
• Bag of Tricks for Adversarial Training (2020, unknown): “…via encoding the mechanisms inspired from other domains, including ensemble learning (Tramèr et al., 2018; Pang et al., 2019), metric learning (Mao et al., 2019; Li et al., 2019; Pang et al., 2020c), generative modeling (Jiang et al., 2018; Pang et al., 2018b; Wang & Yu, 2019; Deng et al.,…”
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks (2020, unknown): “33 8 (Mao et al., 2019) 86.”
• Deep Repulsive Prototypes for Adversarial Robustness (2021, methodology): “Mao et al. (2019) used a triplet loss (inspired by metric learning), where one element of the triplet loss is an adversarial example.”