Adversarial Training for Large Neural Language Models

By Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao. 2020

206 citations recorded by the source corpus.

Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts generalization. In natural language processing (NLP), pre-training large neural language models such as BERT have demonstrated impressive gain in generalization for a variety of tasks, with further improvement from adversarial fine-tuning. However, these models are still vulnerable to adversarial attacks. In this paper, we show that adversarial pre-training can improve both generalization and robustness. We propose a general algorithm ALUM (Adversarial training for large neural LangUage Models), which regularizes the training objective by applying perturbations in the embedding space that maximizes the adversarial loss. We present the first comprehensive study of adversarial training in all stages, including pre-training from scratch, continual pre-training on a well-trained model, and task-specific fine-tuning. ALUM obtains substantial gains over BERT on a wide range of NLP tasks, in both regular and adversarial scenarios. Even for models that have been well trained on extremely large text corpora, such as RoBERTa, ALUM can still produce significant gains from continual pre-training, whereas conventional non-adversarial methods can not. ALUM can be further combined with task-specific fine-tuning to attain additional gains. The ALUM code is publicly available at this https URL.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Zhe Gan
3. Jingjing Liu
4. Mohit Bansal
5. Weizhu Chen
6. Yu Cheng
7. Linjie Li
8. Percy Liang

In Large Language Models

1. Jianfeng Gao
2. Zhe Gan
3. Yu Cheng
4. Weizhu Chen
5. Yen-Chun Chen
6. Linjie Li
7. Pengcheng He
8. Jingjing Liu

In Safety, Robustness & Interpretability

1. Xiaodong Liu
2. Weizhu Chen
3. Jianfeng Gao
4. Pengcheng He
5. Zhe Gan
6. Yu Cheng
7. Chen Zhu
8. Bo Li

Citations in the corpus

• DeBERTa: Decoding-enhanced BERT with Disentangled Attention (2020, background): “Since 2018, we have seen the rise of a set of large-scale Transformer-based Pre-trained Language Models (PLMs), such as GPT [2, 3], BERT [4], RoBERTa [5], XLNet [6], UniLM [7], ELECTRA [8], T5 [9], ALUM [10], StructBERT [11] and ERINE [12] .”
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, background): “Due to growing dominance of large-scale pre-training, very recent work has started to explore adversarial training in the pre-training stage [19, 10, 37].”
• Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data (2020, methodology): “For SciTail, our model matches the current SOTA2 ALUM (Liu et al., 2020), a RoBERTa large based model that additionally uses the SMART (Jiang et al.”
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models (2020, background): “tioned shows that by injecting adversarial perturbation to multimodal embeddings, PGD-based (Projected Gradient Descent) AT [45, 73, 81, 27, 41, 32] can help the model adapt to linguistic variation and visual content manipulation, yielding better model robustness; but with only limited effect (somet”
• GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts (2023, background): “Furthermore, their widespread use has made them targets for adversarial attacks, including backdoor attacks [5, 32, 41], prompt injection [23,36,49], and data poisoning [34,43,73].”
• Prompt Injection attack against LLM-integrated Applications (2023, background): “However, this expanded usage is accompanied by heightened security vulnerabilities, manifested by a broad spectrum of adversarial tactics such as jailbreak [15, 41, 60] and backdoor [7, 36, 68], and complex data poisoning [32, 38, 67].”
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, methodology): “Existing work either uses PGD-based attacks to generate adversarial examples in the embedding space of NLP as data augmentation (Zhu et al., 2020a), or regularizes the standard objective using virtual adversarial training (Jiang et al., 2020; Liu et al., 2020; Gan et al., 2020).”
• Deep Learning--based Text Classification (2020, background): “ALUM [14] introduces an adversarial loss for model pretraining that improves the model’s generalization to new tasks and robustness to adversarial attacks.”
• On Robust Prefix-Tuning for Text Classification (2022, methodology): “…of adversarial training approaches for text classification, in this work, we did not use them for our baselines as most of them (Zhu et al., 2020; Jiang et al., 2020a; Liu et al., 2020; Dong et al., 2021) are based on the KL-divergence regularizer, and also require excessively long training time.”
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding (2020, methodology): “With such a characterization, adversarial training (Zhu et al., 2020; Jiang et al., 2020; Liu et al., 2020; Cheng et al., 2020) also performs label-preserving transformation in embedding space, and thus is considered as an alternative to data augmentation methods in this work.”
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, unknown): “Several prior works have explored adversarial training for LLMs using the smoothness formulation, including (Ebrahimi et al., 2017; Zhu et al., 2019; Liu et al., 2020; Morris et al., 2020).”
• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020, background): “They can also invoke adversarial training (Miyato et al., 2018; Jiang et al., 2019; Liu et al., 2020), which helps improve model robustness and generalizability.”
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, unknown): “However, the vast majority of these defenses, e.g., those that use adversarial training [22, 23] or data augmentation [24], require retraining the underlying model, which is computationally infeasible for LLMs.”
• A Comprehensive Overview of Large Language Models (2023, unknown): “This paper will serve as a valuable resource for researchers, offering insights into the recent advancements in LLMs and providing fundamental concepts and details to develop improved LLMs.”
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing (2020, background): “Adversarial pretraining has been shown to be highly effective in boosting performance in general-domain applications [36].”