Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

By Guanghui Qin, Jason Eisner. 2021

602 citations recorded by the source corpus.

Natural-language prompts have recently been used to coax pretrained language models into performing other AI tasks, using a fill-in-the-blank paradigm (Petroni et al., 2019) or a few-shot extrapolation paradigm (Brown et al., 2020). For example, language models retain factual knowledge from their training corpora that can be extracted by asking them to “fill in the blank” in a sentential prompt. However, where does this prompt come from? We explore the idea of learning prompts by gradient descent—either fine-tuning prompts taken from previous work, or starting from random initialization. Our prompts consist of “soft words,” i.e., continuous vectors that are not necessarily word type embeddings from the language model. Furthermore, for each task, we optimize a mixture of prompts, learning which prompts are most effective and how to ensemble them. Across multiple English LMs and tasks, our approach hugely outperforms previous methods, showing that the implicit factual knowledge in language models was previously underestimated. Moreover, this knowledge is cheap to elicit: random initialization is nearly as good as informed initialization.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Finetuned Language Models Are Zero-Shot Learners (2021, background): “Another line of work that shares this goal prompts models with continuous inputs optimized via backpropagation to substantially improve performance (Li & Liang, 2021; Lester et al., 2021; Qin & Eisner, 2021).”
Large Language Models Are Human-Level Prompt Engineers (2022, methodology): “…et al., 2022; Ziegler et al., 2019), in-context learning (Brown et al., 2020), and several forms of prompt generation (Gao, 2021), including both differentiable tuning of soft prompts (Qin & Eisner, 2021; Lester et al., 2021) and natural language prompt engineering (Reynolds & McDonell, 2021).”
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2021, background): “…2021) Prompt Token Embeds Learned (Continuous) O PTI P ROMPT (Zhong et al., 2021) Prompt Token Embeds Learned (Continuous) Soft Prompts (Qin and Eisner, 2021) All Contextualized Embeds Learned (Continuous) GPT-3 (Brown et al., 2020) None Manual PET (Schick and Schütze, 2021a) All Manual LM-BFF…”
Few-Shot Self-Rationalization with Natural Language Prompts (2021, background): “Alternatively, in few-shot learning, researchers are studying prompts in the form of continuous (or soft) vectors that do not correspond to real tokens (Qin and Eisner, 2021; Li and Liang, 2021).”
QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “…While we use Low-rank Adapters [28] (LoRA), many other Parameter Efficient FineTuning (PEFT) methods have been proposed such as prompt tuning [49, 33, 34], tuning the embedding layer inputs [1], tuning hidden states (IA 3 ) [37], adding full layers [27], tuning biases [71], learning a mask…”
GPT Understands, Too (2021, background): “Results show that P-Tuning can be effectively combined with existing discrete patterns and achieve further performance improvement. cusing on improving knowledge probing (Qin and Eisner, 2021; Zhong et al., 2021).”
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “Currently, prompt tuning is one of the most parameter-efficient methods for large language models [45, 14, 53].”
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts (2022, neutral): ““continuous” or “soft” prompts) not corresponding to words in the model’s original vocabulary (Lester et al., 2021; Qin and Eisner, 2021)”
MAGMA - Multimodal Augmentation of Generative Models through Adapter-based Finetuning (2021, methodology): “Our general approach is an image conditioned variant of soft-prompting or preﬁx tuning [27,36,45] for language transformers and is an extension of the Frozen method introduced in [52].”
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (2023, methodology): “Various PEFT techniques have been explored, including prompt tuning [35, 30, 39, 38, 50, 72], Low-Rank Adaptation (LoRA) [23, 69, 20], and adapters [22, 48, 37, 9, 55].”
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? (2021, background): “…examples with special tokens, optionally initialized with word embeddings; but during learning, those tokens can be updated arbitrarily such that the ﬁnal embeddings often do not correspond to any real word in the vocabulary (e.g., Lester et al., 2021; Li and Liang, 2021; Qin and Eisner, 2021).”
Parameter-Efficient Transfer Learning with Diff Pruning (2020, background): “In recent work, Li & Liang (2021) and Qin & Eisner (2021) explore the use of learned prompts on top of pretrained models to obtain task-specific models.”
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks (2021, background): “The most significant improvement originates from using deep prompt tuning, which is to apply continuous prompts for every layer of the pretrained model (Li and Liang, 2021; Qin and Eisner, 2021).”
Learning To Retrieve Prompts for In-Context Learning (2021, methodology): “…et al., 2019; Schick and Schütze, 2021), decoding meth-ods (Min et al., 2021; Zhao et al., 2021; Holtzman et al., 2021), and methods for extracting either hard (Shin et al., 2020; Haviv et al., 2021) or soft (Li and Liang, 2021; Zhong et al., 2021; Qin and Eisner, 2021) prompts automatically.”
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer (2021, background): “More recent work explores methods for learning soft prompts (Liu et al., 2021b; Qin and Eisner, 2021; Li and Liang, 2021; Lester et al., 2021), which can be seen as additional learnable parameters injected into the language model.”