PADA: A Prompt-based Autoregressive Approach for Adaptation to Unseen Domains

By Eyal Ben-David, Nadav Oved, Roi Reichart. 2021

70 citations recorded by the source corpus.

Subfields: Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

1. Dongju Park
2. Dong-hyun Ham
3. Fei Huang
4. Dan Roth
5. Shumin Deng
6. Eneko Agirre
7. Thien Huu Nguyen
8. Luo Si

Citations in the corpus

• Large Language Models Are Human-Level Prompt Engineers (2022, methodology): “In our paper, we borrow components from discrete prompt search methods, such as prompt generation (Gao et al., 2021; Ben-David et al., 2021), prompt scoring (Davison et al.”
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, background): “…Tam et al., 2021) and continuous or soft prompts (Li & Liang, 2021; Liu et al., 2021c; Lester et al., 2021), ( ii ) applications of PL (Han et al., 2021; Ben-David et al., 2021; Ding et al., 2021), and analysis of prompt-based learning (Liu et al., 2021a; Le Scao & Rush, 2021; Zhong et al., 2021).”
• What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021, background): “The discrete approach optimizes directly on the token space (Ben-David et al., 2021; Shin et al., 2020) and has the advantage of transferability.”
• Black-box Prompt Learning for Pre-trained Language Models (2022, background): “of prompts, the prompt-based learning can be categorized into two kinds: discrete prompt (Wallace et al., 2019; Shin et al., 2020; Jiang et al., 2020; Yuan et al., 2021; Haviv et al., 2021; Gao et al., 2021; Ben-David et al., 2021) and continuous prompt (Zhong et al.”
• Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, methodology): “In PADA, the prompts are properties of source domains, generated based on their relatedness to the input example (from a new domain).”
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction (2021, background): “With appropriate manual prompts, series of studies [6, 31, 35, 37, 40, 41] have been proposed, demonstrating the advancement of prompttuning.”