On Transferability of Prompt Tuning for Natural Language Understanding

By Yusheng Su, Yujia Qin, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Peng Li, Juan-Zi Li, Lei Hou, Maosong Sun, Jie Zhou. 2021

21 citations recorded by the source corpus.

Subfields: Natural Language Processing, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Zhiyuan Liu
2. Maosong Sun
3. Mohit Bansal
4. Yang Liu
5. Peng Li
6. Yankai Lin
7. Caiming Xiong
8. Jie Zhou

In Prompting & Reasoning

1. Zhiyuan Liu
2. Mohit Bansal
3. Maosong Sun
4. Ning Ding
5. Derek Tam
6. Peng Li
7. Shengding Hu
8. Zonghan Yang

Citations in the corpus

• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “Prompt tuning has been applied to various applications for NLP including continual learning [59], model robustness [60, 61], summarization [62], machine translation [63], co-training [64], probing language models [65, 65], inverse prompting [66] and transfer learning [67].”
• ELLE: Efficient Lifelong Pre-training for Emerging Data (2022, background): “Soft prompts have been demonstrated as excellent task indicators (Qin et al., 2021b) and have non-trivial transferability among tasks (Su et al., 2021).”
• Parameter-efficient fine-tuning of large-scale pre-trained language models (2023, methodology): “Moreover, similar to other methods, prompt-tuning also demonstrates transferability across tasks 24,26 , which suggests that appropriate initialization could be substantially beneficial for downstream tasks.”