SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

By Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Matthew Cer. 2021

337 citations recorded by the source corpus.

There has been growing interest in parameter-efficient methods to apply pre-trained language models to downstream tasks. Building on the Prompt Tuning approach of Lester et al. (2021), which learns task-specific soft prompts to condition a frozen pre-trained model to perform different tasks, we propose a novel prompt-based transfer learning approach called SPoT: Soft Prompt Transfer. SPoT first learns a prompt on one or more source tasks and then uses it to initialize the prompt for a target task. We show that SPoT significantly boosts the performance of Prompt Tuning across many tasks. More remarkably, across all model sizes, SPoT matches or outperforms standard Model Tuning (which fine-tunes all model parameters) on the SuperGLUE benchmark, while using up to 27,000× fewer task-specific parameters. To understand where SPoT is most effective, we conduct a large-scale study on task transferability with 26 NLP tasks in 160 combinations, and demonstrate that many tasks can benefit each other via prompt transfer. Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task.

Subfields: Artificial Intelligence, Prompting & Reasoning, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Prompting & Reasoning

In Efficiency & Compression

Citations in the corpus

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “[19] showed that pre-training the prompt embeddings in prompt tuning can improve performance when fine-tuning on downstream few-shot tasks.”
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, background): “Cross-task generalization with PLMs Multitask learning and transfer learning go beyond task boundaries, view different tasks as related, and have been shown to outperform single-task learning (Aghajanyan et al., 2021a; Vu et al., 2021).”
Progressive Prompts: Continual Learning for Language Models (2023, background): “A similar phenomenon has been shown by Vu et al. (2021) – prompts learned on informative source tasks served as a good initialization for other downstream tasks.”
Continual Prompt Tuning for Dialog State Tracking (2022, background): “Gu et al. (2021) and Vu et al. (2021) further 161 explore the transferability of soft prompts across 162 tasks.”
Leveraging QA Datasets to Improve Generative Data Augmentation (2022, background): “Other work proposes to pre-train prompts by adding soft prompts into the pre-training stage (Gu et al., 2022; Vu et al., 2022b,a).”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models (2024, neutral): “In this section, we will introduce important techniques involved in each process.”
Black-box Prompt Learning for Pre-trained Language Models (2022, methodology): “First, we use SST-2 as the source task following Vu et al. (2021) and optimize the discrete prompt tokens with our proposed BDPL.”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, background): “, lower std-dev) accuracy, as our approach does not suffer from the sensitivity to initialization, a common issue of soft prompt tuning methods in few-shot settings (Gu et al., 2021; Vu et al., 2021; Su et al., 2021; Li and Liang, 2021).”
OpenPrompt: An Open-source Framework for Prompt-learning (2021, background): “These practical studies imply that we may use prompts to more effectively and efﬁciently dig the knowledge kept in PLMs, leading to a deeper understanding of the underlying principles of their mechanisms (Wei et al., 2021; Qin et al., 2021; Vu et al., 2021).”
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation (2022, methodology): “…approach has been used to boost performance on English tasks for both M ODEL T UNING (Phang et al., 2019; Vu et al., 2020) and P ROMPT T UNING (Vu et al., 2022), and has been successfully applied to the zero-shot cross-lingual transfer setting (Phang et al., 2020; Maurya et al., 2021) for M…”
Towards Agile Text Classifiers for Everyone (2023, background): “…model parameter updates —and parameter-efficient tuning (PET), a branch of transfer learning that adapts only a small number of parameters for new tasks and has been shown to obtain performance comparable to fine-tuning all parameters (Li and Liang, 2021a; Lester et al., 2021; Vu et al., 2022a).”
Editing Models with Task Arithmetic (2022, positive): “[91; 92], inter alia, explored strategies for representing tasks with continuous embeddings, in order to to predict task similarities and transferability, or to create taxonomic relations.”
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation (2023, background): “The success of adapters in NLP has also led to widespread interest in applying adapter techniques to visual tasks.”
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning (2022, background): “Vu et al. (2021) learned task speciﬁc prompts that can ben-eﬁt each other via prompt transfer.”
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models (2024, neutral): “PEFT encompasses strategies like partial fine-tuning [15, 16, 17, 18, 19, 20, 21, 22], soft prompt fine-tuning [23, 24, 25, 26, 27, 28, 29], non-linear adapter fine-tuning [30, 31, 32, 33, 34, 35, 36, 37, 38, 39], and low rank adapter based fine-tuning [40, 41, 11, 42].”
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment (2023, methodology): “Several works [26], [27], [28] employ prompt-tuning for multi-task learning.”