Prefix-Tuning: Optimizing Continuous Prompts for Generation
By Xiang Lisa Li, Percy Liang. 2021
6,083 citations recorded by the source corpus.
Fine-tuning is the de facto way of leveraging large pretrained language models for downstream tasks. However, fine-tuning modifies all the language model parameters and therefore necessitates storing a full copy for each task. In this paper, we propose prefix-tuning, a lightweight alternative to fine-tuning for natural language generation tasks, which keeps language model parameters frozen and instead optimizes a sequence of continuous task-specific vectors, which we call the prefix. Prefix-tuning draws inspiration from prompting for language models, allowing subsequent tokens to attend to this prefix as if it were “virtual tokens”. We apply prefix-tuning to GPT-2 for table-to-text generation and to BART for summarization. We show that by learning only 0.1% of the parameters, prefix-tuning obtains comparable performance in the full data setting, outperforms fine-tuning in low-data settings, and extrapolates better to examples with topics that are unseen during training.
Subfields: Artificial Intelligence, Natural Language Processing, Efficiency & Compression
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- Yejin Choi
- Quoc V. Le
- Noah A. Smith
In Natural Language Processing
- Mohit Bansal
- Graham Neubig
- Luke S. Zettlemoyer
- Yejin Choi
- Jianfeng Gao
- Percy Liang
- Hannaneh Hajishirzi
- Noah A. Smith
In Efficiency & Compression
- Luke S. Zettlemoyer
- Mohit Bansal
- Barret Zoph
- Quoc V. Le
- Dacheng Tao
- Michael Lewis
- Yi-Lin Sung
- Jaemin Cho
Citations in the corpus
- Few-shot Learning with Retrieval Augmented Language Models (2022, neutral): “, 2021), or learning “soft-prompts” (Lester et al., 2021; Li and Liang, 2021).”
- Efficient Large Scale Language Modeling with Mixtures of Experts (2021, background): “of a small number of additional task-specific parameters instead (Lester et al., 2021; Li and Liang, 2021; Houlsby et al., 2019).”
- Program Synthesis with Large Language Models (2021, positive): “These results suggest that methods such as prompt-tuning [Li and Liang, 2021, Lester et al., 2021] could yield substantial performance improvements in this domain.”
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (2021, background): “optimization of prompts [17, 22], and task-specific adapter networks [25].”
- OPT: Open Pre-trained Transformer Language Models (2022, background): “tr11-176B-ml-logs/tensorboard has also been work on eliciting prompting behavior in smaller models (Schick and Schütze, 2020; Gao et al., 2021b; Li and Liang, 2021; Lester et al., 2021; Scao and Rush, 2021), improving the flexibility of prompting (Shin et al.”
- Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “Li and Liang (2021) added a “prefix” before real inputs and searched the optimal soft prompt in the embedding layer in models by gradient.”
- Efficient Memory Management for Large Language Model Serving with PagedAttention (2023, background): “Moreover, the shared prefix can be further tuned, via prompt engineering, to improve the accuracy of the downstream tasks [26, 27].”
- Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, background): “Li and Liang (2021) focus on left-to-right LMs for generation tasks, whereas we focus on masked LMs for clas-sification tasks.”
- Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “…prefixes, e.g., dynamically selecting good “training” examples based on the question (Liu et al., 2021a), or even representing the prefix latently (Li and Liang, 2021), our work elaborates the input x itself to clarify the intended task, by adding user feedback fb from previous misunderstandings.”
- Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Namely, we show that larger models generate notably more plausible explanations, and “prefix tuning” (Li and Liang, 2021) has been show to learn two condition generation tasks using only 0.”
- Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, background): “Prompt-tuning [44], [45] may work particularly well with representations of this kind, potentially enabling more rapid adaptation to new tasks [46].”
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, methodology): “, “I love this movie, it is a [Z] movie”), and prefix prompts [67, 71], which continue a string prefix (e.”
- Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “The prompts can be either in natural language [57] or as learnable parameters [31].”
- LoRA: Low-Rank Adaptation of Large Language Models (2021, methodology): “We focus on “prefixing”, which prepends these tokens to the prompt, and “infixing”, which appends to the prompt; both are discussed in [21].”
- QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “…we use Low-rank Adapters [28] (LoRA), many other Parameter Efficient FineTuning (PEFT) methods have been proposed such as prompt tuning [49, 33, 34], tuning the embedding layer inputs [1], tuning hidden states (IA 3 ) [37], adding full layers [27], tuning biases [71], learning a mask over…”
- Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “These include adding small adapter modules [43], fine-tuning a small part of the LM [141], showing in-context examples in the prompt [11], or optimizing the prompt [56, 60] through gradient descent.”