SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
By Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao. 2019
613 citations recorded by the source corpus.
Transfer learning has fundamentally changed the landscape of natural language processing (NLP). Many state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely high complexity of pre-trained models, aggressive fine-tuning often causes the fine-tuned model to overfit the training data of downstream tasks and fail to generalize to unseen data. To address such an issue in a principled manner, we propose a new learning framework for robust and efficient fine-tuning for pre-trained models to attain better generalization performance. The proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the complexity of the model; 2. Bregman proximal point optimization, which is an instance of trust-region methods and can prevent aggressive updating. Our experiments show that the proposed framework achieves new state-of-the-art performance on a number of NLP tasks including GLUE, SNLI, SciTail and ANLI. Moreover, it also outperforms the state-of-the-art T5 model, which is the largest pre-trained model containing 11 billion parameters, on GLUE.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Alignment & RLHF, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Yejin Choi
- Hannaneh Hajishirzi
- Caiming Xiong
- Graham Neubig
In Computer Vision and Pattern Recognition
- Jianfeng Gao
- Luc Van Gool
- Dacheng Tao
- Mohit Bansal
- Trevor Darrell
- Xiaogang Wang
- Cordelia Schmid
- Ming-Hsuan Yang
In Alignment & RLHF
- Xiaodong Liu
- Jianfeng Gao
- Mari Ostendorf
- Noah A. Smith
- Luke S. Zettlemoyer
- Diyi Yang
- Hannaneh Hajishirzi
- Weijia Shi
In Optimization & Training
- Xiaodong Liu
- Jianfeng Gao
- Weizhu Chen
- Tian Li
- Virginia Smith
- Luke S. Zettlemoyer
- Mohit Bansal
- Pengcheng He
Citations in the corpus
- Adversarial Training for Large Neural Language Models (2020, methodology): “Namely, x is the sub-word embedding in f ( x ; θ ) (Jiang et al., 2019; Zhu et al., 2019).”
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, methodology): “There are also some parallel studies on applying AT to language modeling [67] and natural language understanding [45, 85, 24].”
- Few-shot Mining of Naturally Occurring Inputs and Outputs (2022, background): “Likewise, regularization based approaches have also been shown to improve performance (Jiang et al., 2020; Agha-janyan et al., 2020).”
- Better Fine-Tuning by Reducing Representational Collapse (2020, methodology): “Consistent with prior work (Jiang et al., 2019; Zhu et al., 2019), we focus on improving the performance of RoBERTa-Large based models in the single task setting (Liu et al., 2019).”
- UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021, methodology): “…first, we add a group of attention bias parameters into the decoder cross-attention block to feature the ranking information of retrieved contexts; second, we add the adversarial training (Ju et al., 2019; Jiang et al., 2020; Pereira et al., 2021) to improve the model generalization ability.”
- Robust fine-tuning of zero-shot models (2021, methodology): “In natural language processing, previous work proposed stable fine-tuning methods that incur computational overhead [45, 111], alleviating problems such as representational collapse [1].”
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning (2020, methodology): “To improve the generalization performance, Jiang et al. (2020) propose a local smoothness-inducing regularizer to manage the complexity of the model and a Bregman proximal point optimization method, an instance of trust-region methods, to prevent aggressive updating of the model during fine-tuning.”
- Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data (2020, methodology): “Unlike MT-DNN SMART however, we do not add context from a semantic similarity model – MT-DNN SMART is built off HNN (He et al., 2019).”
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models (2020, background): “tioned shows that by injecting adversarial perturbation to multimodal embeddings, PGD-based (Projected Gradient Descent) AT [45, 73, 81, 27, 41, 32] can help the model adapt to linguistic variation and visual content manipulation, yielding better model robustness; but with only limited effect (somet”
- STraTA: Self-Training with Task Augmentation for Better Few-shot Learning (2021, neutral): “…et al., 2020), intermediate-task fine-tuning (Phang et al., 2019), multi-task pre-finetuning (Aghajanyan et al., 2021a), better de-sign choices and training strategies (Mosbach et al., 2021; Zhang et al., 2021), and regularization-oriented techniques (Jiang et al., 2020; Aghajanyan et al., 2021b).”
- An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, methodology): “Hiddenspace perturbations augment existing data by adding perturbations to the hidden representations of tokens (Miyato et al., 2017; Zhu et al., 2020; Jiang et al., 2019; Chen et al., 2020d; Shen et al., 2020; Chen et al., 2021) or sentences (Hsu et al.”
- Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021, methodology): “Some of them utilize the pretrained weights to regularize the deviation of the fine-tuned model (Lee et al., 2020; Daumé III, 2007; Chen et al., 2020), while others compress the output information (Mahabadi et al., 2021) or injects noise into the input (Jiang et al., 2020; Aghajanyan et al., 2021).”
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Jin et al. [121] conducted a comprehensive study on LLMs’ understanding of binary code semantics [122] across different architectures and optimization levels, providing key insights for future research in this area.”
- Taming Sparsely Activated Transformer with Stochastic Experts (2021, methodology): “• SMART (Jiang et al., 2020) utilizes a smoothness inducing adversarial regularizer to penalize the worst case difference between predictions of a clean input and a perturbed input.”