To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks

By Matthew E. Peters, Sebastian Ruder, Noah A. Smith. 2019

485 citations recorded by the source corpus.

While most previous work has focused on different pretraining objectives and architectures for transfer learning, we ask how to best adapt the pretrained model to a given target task. We focus on the two most common forms of adaptation, feature extraction (where the pretrained weights are frozen), and directly fine-tuning the pretrained model. Our empirical results across diverse NLP tasks with two state-of-the-art models show that the relative performance of fine-tuning vs. feature extraction depends on the similarity of the pretraining and target tasks. We explore possible explanations for this finding and provide a set of adaptation guidelines for the NLP practitioner.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Prompting & Reasoning

In Representation & Self-Supervised Learning

Citations in the corpus

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, background): “, 2019b, 2018; Conneau and Kiela, 2018], fine-tuning methods [Howard and Ruder, 2018; Houlsby et al., 2019; Peters et al., 2019], and more.”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, background): “The successes of sentence encoder pretraining have sparked a line of work analyzing these models (Zhang and Bowman, 2018; Peters et al., 2018b; Tenney et al., 2019b; Peters et al., 2019; Tenney et al., 2019a; Liu et al., 2019, i.a.).”
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, methodology): “[98]) ine-tuned as well to make it more suitable for solving the downstream task.”
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers (2021, background): “Additionally, fine-tuning of over-parameterized models on low-resource datasets has been shown to be subject to instabilities and may lead to poor performance [6, 7].”
WARP: Word-level Adversarial ReProgramming (2021, background): “The latter can some-times yield better results (Peters et al., 2019), while the ﬁrst one usually offers better stability for smaller datasets.”
AdapterFusion: Non-Destructive Task Composition for Transfer Learning (2020, methodology): “Transfer to a task of interest is achieved by ﬁnetuning all the weights of the pretrained model on that single task , often yielding state-of-the-art re-sults (Zhang and Yang, 2017; Ruder, 2017; Howard and Ruder, 2018; Peters et al., 2019) .”
Training Neural Networks with Fixed Sparse Masks (2021, background): “Standard practice is to update all of the model’s parameters during ﬁne-tuning, though in some cases reasonable performance can be attained by only ﬁne-tuning the output layer of the model [20, 8, 38].”
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020, background): “…pretraining (hence, expense of more rounds of pretraining), relevance to a data sample of a given task, and transferability to 4In contrast, Peters et al. (2019) find that the JensenShannon divergence on term distributions between BERT’s pretraining corpora and each MULTINLI domain…”
Show Your Work: Improved Reporting of Experimental Results (2019, methodology): “Peters et al. (2019) found that feature extraction is preferable to ﬁne-tuning ELMo embeddings.”
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings (2019, methodology): “Peters et al. (2019) and Liu et al. (2019) conﬁrmed the viability of using ELMo representations directly as features for downstream tasks without re-training the full model on the target task.”
Linguistic Knowledge and Transferability of Contextual Representations (2019, background): “However, such end-task– speciﬁc contextualization can come from either ﬁne-tuning CWR s or using ﬁxed output features as inputs to a task-trained contextualizer; Peters et al. (2019) begins to explore when each approach should be applied.”
Revisiting Few-sample BERT Fine-tuning (2020, methodology): “BERT Representation Transferability BERT pre-trained representations have been widely studied using probing methods showing that the pre-trained features from intermediate layers are more transferable [15, 16, 25, 40, 41] or applicable [55] to new tasks than features from later layers, which change ”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “To the best of our knowledge, there has been no work systematically analyzing the effect of generic pre-trained initialization on catastrophic forgetting in LL scenarios.”
Visualizing and Understanding the Effectiveness of BERT (2019, methodology): “Pre-trained contextualized word representations learned from language modeling objectives, such as CoVe (McCann et al., 2017), ELMo (Peters et al., 2018), ULMFit (Howard and Ruder, 2018b), GPT (Radford et al., 2018, 2019), and BERT (Devlin et al., 2018), have shown strong performance on a variety of”
AdapterHub: A Framework for Adapting Transformers (2020, methodology): “Prior to the introduction of adapters, in order to achieve SotA performance on downstream tasks, the entire pre-trained transformer model needs to be fine-tuned (Peters et al., 2019).”
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks (2019, background): “Peters et al. (2019) explore whether it is preferable to fine-tune the entire model on a specific task or to use the learned representations as features, i.e. freezing the pre-trained model.”
Pre-trained models for natural language processing: A survey (2020, background): “tion way, the pretrained models are regarded as o-the-shelf feature extractors. Moreover, it is important to expose the internal layers as they typically encode the most transferable representations [124]. Although both these two ways can signiﬁcantly beneﬁt most of NLP tasks, feature extraction way”
Measuring and Reducing Gendered Correlations in Pre-trained Models (2020, positive): “…checkpoint is a strategy for reducing correlations, adding another factor into what to consider when deciding how to use a pre-trained model (either as a feature extractor or via fune-tuning), on top of similarity between pre-training and application task, identiﬁed in Peters et al. (2019).”