PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains

By Eyal Ben-David, Nadav Oved, Roi Reichart. 2021

103 citations recorded by the source corpus.

Natural Language Processing algorithms have made incredible progress, but they still struggle when applied to out-of-distribution examples. We address a challenging and underexplored version of this domain adaptation problem, where an algorithm is trained on several source domains, and then applied to examples from unseen domains that are unknown at training time. Particularly, no examples, labeled or unlabeled, or any other knowledge about the target domain are available to the algorithm at training time. We present PADA: An example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation, based on the T5 language model. Given a test example, PADA first generates a unique prompt for it and then, conditioned on this prompt, labels the example with respect to the NLP prediction task. PADA is trained to generate a prompt that is a token sequence of unrestricted length, consisting of Domain Related Features (DRFs) that characterize each of the source domains. Intuitively, the generated prompt is a unique signature that maps the test example to a semantic space spanned by the source domains. In experiments with 3 tasks (text classification and sequence tagging), for a total of 14 multi-source adaptation scenarios, PADA substantially outperforms strong baselines.1

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pengfei Liu
2. Graham Neubig
3. Weizhe Yuan
4. Jinlan Fu
5. Zhengbao Jiang
6. Hiroaki Hayashi
7. Yixin Liu
8. Yang Liu

In Computer Vision and Pattern Recognition

1. Zhengbao Jiang
2. Pengfei Liu
3. Jinlan Fu
4. Graham Neubig
5. Weizhe Yuan
6. Hiroaki Hayashi
7. Felix Stahlberg
8. Orhan Firat

In Prompting & Reasoning

1. Graham Neubig
2. Pengfei Liu
3. Jinlan Fu
4. Zhengbao Jiang
5. Weizhe Yuan
6. Hiroaki Hayashi
7. Yang Liu

Citations in the corpus

• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, methodology): “[5] use self-generated domain related features (DRFs) to augment the original text input and perform sequence tagging as a sequence-to-sequence problem using a seq2seq pre-trained LM.”
• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, unknown): “The TTA paradigm is also studied in tasks of the NLP field, such as reading comprehension [430], question answering [527], [584], sentiment analysis [245], [585], entity recognition [376], and aspect prediction [453], [586].”