TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

By Chengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang, Ming Gao. 2021

65 citations recorded by the source corpus.

Recent studies have shown that prompts improve the performance of large pre-trained language models for few-shot text classification. Yet, it is unclear how the prompting knowledge can be transferred across similar NLP tasks for the purpose of mutual reinforcement. Based on continuous prompt embeddings, we propose TransPrompt, a transferable prompting framework for few-shot learning across similar tasks. In TransPrompt, we employ a multi-task meta-knowledge acquisition procedure to train a meta-learner that captures cross-task transferable knowledge. Two de-biasing techniques are further designed to make it more task-agnostic and unbiased towards any tasks. After that, the meta-learner can be adapted to target tasks with high accuracy. Extensive experiments show that TransPrompt outperforms single-task and cross-task strong baselines over multiple NLP tasks and datasets. We further show that the meta-learner can effectively improve the performance on previously unseen tasks; and TransPrompt also outperforms strong fine-tuning baselines when learning with full training sets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lifu Huang
2. Minqian Liu
3. Shiyu Chang

Citations in the corpus

• Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “Lifelong Event Detection Deep neural networks have shown state-of-the-art performance on supervised event detection (Nguyen et al., 2016; Feng et al., 2016; Zhang et al., 2017; Huang and Ji, 2020; Wang et al., 2021b).”
• Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “With the rise of LLMs, prompting-based methods [71, 245, 260, 313] achieved notable results by leveraging LLMs’ internal knowledge.”