FreeLB: Enhanced Adversarial Training for Natural Language Understanding

By Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, Jingjing Liu. 2019

520 citations recorded by the source corpus.

Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformer-based models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44\% and 67.75\% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well. Code is available at \url{this https URL .

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

UnifiedQA: Crossing Format Boundaries With a Single QA System (2020, background): “(Mitra et al., 2020) RoBERTa (Clark et al.,2019c) FreeLB-RoBERTa (Zhu et al., 2020) RoBERTa (Clark et al.,2019c) FreeLB-RoBERTa (Zhu et al., 2020) -- KF+SIR +2Step (Mitra et al., 2020) 75.7 (Lan et al.,2019) FreeLB-RoBERTa (Zhu et al.,2020) RoBERTa (Sakaguchi et al.,2019) RoBERTa (Bisk et al.,…”
Adversarial Training for Large Neural Language Models (2020, methodology): “Namely, x is the sub-word embedding in f ( x ; θ ) (Jiang et al., 2019; Zhu et al., 2019).”
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping (2020, background): “Despite its simplicity and ubiquity in modern NLP, this process has been shown to be brittle (Devlin et al., 2019; Phang et al., 2018; Zhu et al., 2019; Raffe et al., 2019), where ﬁne-tuning performance can vary substantially across different training episodes, even with ﬁxed hyperparameter values.”
A Primer in BERTology: What We Know About How BERT Works (2020, background): “• Adversarial token perturbations improve robustness of the model (Zhu et al., 2019).”
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines (2020, background): “Despite the ubiquity of these hypotheses (Devlin et al., 2019; Phang et al., 2018; Dodge et al., 2020; Lee et al., 2020; Zhu et al., 2020), we argue that that none of them can fully explain the ﬁne-tuning instability.”
Better Fine-Tuning by Reducing Representational Collapse (2020, methodology): “While prior work in non-standard finetuning methods tends to focus on sentence prediction and GLUE tasks (Jiang et al., 2019; Zhu et al., 2019; Zhang et al., 2020), we look to improve abstractive summarization, due to its additional complexity and computational cost, specifically we look at three da”
Pre-trained models for natural language processing: A survey (2020, background): “d make them immune against adversarial attack. Adversarial training aims to improve the generalization by minimizes the maximal risk for label-preserving perturbations in embedding space. Recent work [220, 115] showed that adversarial pre-training or ﬁne-tuning can improve both generalization and ro”
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2019, methodology): “Comparing with the current state-of-the-art adversarial training model, FreeLB, SMART outperforms it on 6 GLUE tasks out of a total of 8 tasks (MNLI, RTE, QNLI, MRPC, SST-2 and STS-B) showing the effectiveness of our model.”
Robust fine-tuning of zero-shot models (2021, methodology): “In natural language processing, previous work proposed stable fine-tuning methods that incur computational overhead [45, 111], alleviating problems such as representational collapse [1].”
Explanations for CommonsenseQA: New Dataset and Models (2021, background): “…goal behind original CQA task is conﬁned only till answering the questions and hence almost all the submissions (Ma et al., 2019; Khashabi et al., 2020; Zhu et al., 2020; Yang et al., 2020) to the leader-board of the CQA dataset focus just on answering the question and not generating explanations.”
G-DAug: Generative Data Augmentation for Commonsense Reasoning (2020, methodology): “6 For COMMONSENSEQA, G-DAUGCombo outperforms the prior non-ensemble stateof-the-art (Zhu et al., 2020) by 0.”
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021, unknown): “As stated in the observations in Section 2, we explore strategies that could improve the sensitivity of PLMs.”
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models (2021, methodology): “To address this challenge, various methods [23, 61, 51, 30] have been proposed to improve the adversarial robustness of language models.”
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective (2020, methodology): “To generate robust representation, Zhu et al. (2020b) formalizes the problem from a mutual-information perspective, which essentially performs adversarial training for worst-case perturbation, while mainly considers the continuous space in computer vision.”
Understanding and Improving Fast Adversarial Training (2020, background): “Adversarially robust models have better interpretability properties [42, 32] and can generalize better [51, 4] including also improved performance under some distribution shifts [48] (although on some performing worse, see [39]).”
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models (2020, background): “tioned shows that by injecting adversarial perturbation to multimodal embeddings, PGD-based (Projected Gradient Descent) AT [45, 73, 81, 27, 41, 32] can help the model adapt to linguistic variation and visual content manipulation, yielding better model robustness; but with only limited effect (somet”
On Robust Prefix-Tuning for Text Classification (2022, methodology): “…are other variants of adversarial training approaches for text classiﬁcation, in this work, we did not use them for our baselines as most of them (Zhu et al., 2020; Jiang et al., 2020a; Liu et al., 2020; Dong et al., 2021) are based on the KL-divergence regularizer, and also require excessively…”