The Power of Scale for Parameter-Efficient Prompt Tuning
By Brian Lester, Rami Al-Rfou, Noah Constant. 2021
5,785 citations recorded by the source corpus.
In this work, we explore “prompt tuning,” a simple yet effective mechanism for learning “soft prompts” to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3’s few-shot learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method “closes the gap” and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant because large models are costly to share and serve and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed “prefix tuning” of Li and Liang (2021) and we provide a comparison to this and other similar approaches. Finally, we show that conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer and enables efficient “prompt ensembling.” We release code and model checkpoints to reproduce our experiments.
Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Quoc V. Le
- Graham Neubig
- Yejin Choi
- Yoshua Bengio
In Large Language Models
- Luke S. Zettlemoyer
- Hinrich Schütze
- Yejin Choi
- Mohit Bansal
- Hannaneh Hajishirzi
- Timo Schick
- Jianfeng Gao
- Michael Lewis
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Swaroop Mishra
- Denny Zhou
- Timo Schick
- Yejin Choi
- Hinrich Schütze
- Mohit Bansal
Citations in the corpus
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, background): “(2020), several general approaches have improved the prompting ability of models, such as automatically learning prompts (Lester et al., 2021) or giving models instructions describing a task (Sanh et al.”
- Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, methodology): “The effectiveness of this method was further enhanced by recent model adaptation work such as prompt tuning [33], which efficiently adapts large language models to individual tasks with robust task performance.”
- Efficient Large Scale Language Modeling with Mixtures of Experts (2021, background): “of a small number of additional task-specific parameters instead (Lester et al., 2021; Li and Liang, 2021; Houlsby et al., 2019).”
- Finetuned Language Models Are Zero-Shot Learners (2021, methodology): “…to instructions, it follows that, if FLAN is indeed more amenable to performing NLP tasks, then it should also achieve better performance when performing inference using soft prompts, represented by prepended continuous variables optimized via prompt tuning (Li & Liang, 2021; Lester et al., 2021).”
- Program Synthesis with Large Language Models (2021, positive): “These results suggest that methods such as prompt-tuning [Li and Liang, 2021, Lester et al., 2021] could yield substantial performance improvements in this domain.”
- OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “Sanh et al. (2022) fine-tune an LM adapted version of T5 11B(Raffel et al., 2020; Lester et al., 2021) on 50 datasets from PromptSource (called T0) and evaluate on a set of 11 held-out tasks which are part of their 4 fully held-out categories.”
- OPT: Open Pre-trained Transformer Language Models (2022, background): “There has also been work on eliciting prompting behavior in smaller models (Schick and Schütze, 2020; Gao et al., 2021b; Li and Liang, 2021; Lester et al., 2021; Scao and Rush, 2021), improving the flexi-bility of prompting (Shin et al., 2020), and understanding why and how prompting works (Liu et…”
- Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, methodology): “We evaluate T5-LM (Lester et al., 2021; Raffel et al., 2020) and GPT3 (Brown et al., 2020) as the vanilla LM baselines (only pretraining, no additional finetuning).”
- Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor (2022, methodology): “We use the same set of hyperparameters for finetuning experiments with T5-LM (Raffel et al., 2020; Lester et al., 2021).”
- Large Language Models Are Human-Level Prompt Engineers (2022, methodology): “…et al., 2022; Ziegler et al., 2019), in-context learning (Brown et al., 2020), and several forms of prompt generation (Gao, 2021), including both differentiable tuning of soft prompts (Qin & Eisner, 2021; Lester et al., 2021) and natural language prompt engineering (Reynolds & McDonell, 2021).”
- Scaling Instruction-Finetuned Language Models (2022, methodology): “For T5 models without instruction finetuning, we use LM-adapted models, which were produced by training T5 on 100B additional tokens from C4 on a standard language modeling objective (Lester et al., 2021).”
- A General Language Assistant as a Laboratory for Alignment (2021, methodology): “Some other observations about prompting and the dependence of prompt-tuning on scale were made in [LARC21] though we did not utilize prompt tuning.”
- Efficient Memory Management for Large Language Model Serving with PagedAttention (2023, background): “Moreover, the shared prefix can be further tuned, via prompt engineering, to improve the accuracy of the downstream tasks [26, 27].”
- Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, methodology): “• Prompt Tuning : Inspired by Liu et al. (2021) and Lester et al. (2021), we use the pattern from Manual Prompt (Prior) but randomly initialize the embeddings of the pattern tokens and learn them using gradient-based optimization.”
- Prefix-Tuning: Optimizing Continuous Prompts for Generation (2021, methodology): “In a followup work, Prompt-tuning (Lester et al., 2021) simplifies our approach and applies it to T5 (Raffel et al., 2020), demonstrating that the performance gap between fine-tuning and p*tuning vanishes as the model size grows.”
- UL2: Unifying Language Learning Paradigms (2022, methodology): “Finally, we conduct additional few-shot in-context one-shot learning using the XSum dataset We compare our model with the baseline T5-XXL, T5-XXL with LM Adaptation (Lester et al., 2021), LaMDA 137B (Thoppilan et al., 2022), and PaLM (8B, 62B, 540B) (Chowdhery et al., 2022).”
- STaR: Bootstrapping Reasoning With Reasoning (2022, background): “Some have even found that replacing few-shot prompts with a “soft prompt” which can be optimized in embedding space results in noticeable gains [15].”