Making Pre-trained Language Models Better Few-shot Learners
By Tianyu Gao, Adam Fisch, Danqi Chen. 2021
2,303 citations recorded by the source corpus.
The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot performance solely by leveraging a natural-language prompt and a few task demonstrations as input context. Inspired by their findings, we study few-shot learning in a more practical scenario, where we use smaller language models for which fine-tuning is computationally efficient. We present LM-BFF—better few-shot fine-tuning of language models—a suite of simple and complementary techniques for fine-tuning language models on a small number of annotated examples. Our approach includes (1) prompt-based fine-tuning together with a novel pipeline for automating prompt generation; and (2) a refined strategy for dynamically and selectively incorporating demonstrations into each context. Finally, we present a systematic evaluation for analyzing few-shot performance on a range of NLP tasks, including classification and regression. Our experiments demonstrate that our methods combine to dramatically outperform standard fine-tuning procedures in this low resource setting, achieving up to 30% absolute improvement, and 11% on average across all tasks. Our approach makes minimal assumptions on task resources and domain expertise, and hence constitutes a strong task-agnostic method for few-shot learning.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Percy Liang
- Yejin Choi
- Graham Neubig
- Hannaneh Hajishirzi
- Noah A. Smith
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Ming-Hsuan Yang
- Mohit Bansal
- Jitendra Malik
- Andrew Zisserman
In Large Language Models
- Luke S. Zettlemoyer
- Hinrich Schütze
- Yejin Choi
- Hannaneh Hajishirzi
- Timo Schick
- Mohit Bansal
- Noah A. Smith
- Michael Lewis
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Timo Schick
- Hinrich Schütze
- Michael Lewis
- Yejin Choi
- Denny Zhou
- Percy Liang
Citations in the corpus
- Few-shot Learning with Retrieval Augmented Language Models (2022, background): “…can also be accomplished by combining textual templates (“prompts”) and various forms of model finetuning, either fully updating a model’s parameters, e.g. for classification (Schick & Schütze, 2021a; Schick & Schutze, 2021; Gao et al., 2021; Tam et al., 2021) or generation (Schick & Schütze, 2021b).”
- Learning Transferable Visual Models From Natural Language Supervision (2021, background): “Similar to the “prompt engineering” discussion around GPT3 (Brown et al., 2020; Gao et al., 2020), we have also observed that zero-shot performance can be significantly improved by customizing the prompt text to each task.”
- Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “, 2021) show that model performance could be sensitive to the prompt order, and (2) multiple sets of prompts (Gao et al., 2021).”
- Finetuned Language Models Are Zero-Shot Learners (2021, background): “Schick & Schütze (2021) (also see Gao et al., 2021; Tam et al., 2021) use task descriptions in cloze-style phrases to help language models assign soft labels for few-shot and semi-supervised learning, though this line of work finetunes new checkpoints for each downstream task.”
- Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, methodology): “The LAMA dataset is one which is often used to evaluate zero-shot performance and GLUE datasets for few-shot performance (Gao et al., 2021), and perhaps this practice should be considered carefully.”
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (2021, background): “Past work has also explored improving the few-shot paradigm by dynamically selecting the most relevant examples for each task [9, 18].”
- Emergent Abilities of Large Language Models (2022, neutral): “Namely, cross-entropy loss improves even for small model scales where the downstream metrics (exact match, BLEU, and accuracy) are close to random and do not improve, which shows that improvements in the log-likelihood of the target sequence can be masked by such downstream metrics.”
- OPT: Open Pre-trained Transformer Language Models (2022, methodology): “The pre-training corpus contains a concatenation of datasets used in RoBERTa (Liu et al., 2019b), the Pile (Gao et al., 2021a), and PushShift.io Red-dit (Baumgartner et al., 2020; Roller et al., 2021).”
- Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, methodology): “We find that, when us-ing prompt-based finetuning (Schick and Schütze, 2021a; Gao et al., 2021), the prompt requires less optimization than previously thought; in fact, the pattern and training examples can be completely cut out (e.g., Figure 1, right).”
- Improving and Simplifying Pattern Exploiting Training (2021, methodology): “This corroborates the finding in Gao et al. (2021) where sometimes the best performing model performs better than ensembling across patterns.”
- Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “Rule reward model Our Rule RM estimates the probability that the rule was violated in a given dialogue, and is trained via a version of instruction-tuning, similar to Gao et al. (2020).”
- Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Since finding appropriate prompts is often challenging (Gao et al., 2021), we then extensively explored natural language prompts for few-shot self-rationalization.”
- Calibrate Before Use: Improving Few-Shot Performance of Language Models (2021, background): “Nevertheless, if these disadvantages are acceptable or avoidable, finetuning can improve accuracy over in-context learning in some cases (Schick & Schütze, 2021b; Gao et al., 2021).”
- Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2022, neutral): “This approach bears resemblance to several recent works (Poesia et al., 2022; Gao et al., 2020; Liu et al., 2021; Rubin et al., 2021).”
- Language Models are Few-Shot Butlers (2021, background): “Moreover, the extensively pretrained language/action modeling head is kept during reinforcement learning instead of initializing a new RL-specific head from scratch, which was shown to lead to better performance in NLP tasks (Gao et al., 2021).”