Few-Shot Text Generation with Pattern-Exploiting Training

By Timo Schick, Hinrich Schütze. 2020

115 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, positive): “We build models using pre-trained LMs with encoder-decoder architectures (GPT3 (Brown et al., 2020) and BART (Lewis et al., 2019)).”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, neutral): “A range of recent works are based on the general idea of models revising their behavior according to changes in their input (Wallace et al., 2019; Gardner et al., 2020; Emelin et al., 2020; Ye and Ren, 2021; Schick and Schütze, 2020; Sheng et al., 2020).”
Language Models are Few-Shot Butlers (2021, methodology): “While we investigated learning under the standard ﬁne-tuning paradigm, more sophisticated approaches could be explored (Schick and Schütze, 2020) and recent works (Brown et al., 2020; Zhao et al., 2021) even suggest that scaled-up and care-fully calibrated models achieve great downstream results…”
True Few-Shot Learning with Language Models (2021, background): “Other work claims to use no validation set for hyperparameter selection [3, 11, 14] but does not describe how they design other aspects of their learning algorithm (e.g., training objectives).”
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP (2021, methodology): “It is also similar to prompt- or keyword-based approaches for controllable text generation (Keskar et al., 2019; Schick and Schütze, 2020; He et al., 2020) but these approaches (i) require either a customized pretraining objective or labeled training data, and (ii) use natural language prompts to in”
ConTinTin: Continual Learning from Task Instructions (2022, background): “Representative work includes (Radford et al., 2019; Schick and Schütze, 2020, 2021), etc. (Please refer to the survey (Liu et al., 2021) for more details.)”
Reframing Instructional Prompts to GPTk’s Language (2021, background): “Discrete Prompts Constructing effective discrete prompts for language models to perform NLP tasks is an active area of research (Schick and Schütze, 2020; Le Scao and Rush, 2021; Tam et al., 2021; Logan IV et al., 2021).”
BARTScore: Evaluating Generated Text as Text Generation (2021, background): “(iii) BARTSCORE supports prompt-based learning [50, 53] which hasn’t been examined in PRISM.”
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, methodology): “Here, a ixed template is applied to every training and test example, as in the PET-TC [166], PET-Gen [165] and LM-BFF [51] models.”
True Few-Shot Learning with Prompts—A Real-World Perspective (2021, methodology): “We brieﬂy review pattern-exploiting training (P ET ) (Schick and Schütze, 2021a,c), the method we use for instruction-based text classiﬁcation.”
Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation (2021, methodology): “with the tasks where prompt-based methods are well applied (Schick and Schütze, 2020; Gao et al., 2021), GDG tasks have several unique characteristics, which may require special attention.”
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems (2021, background): “Discrete prompting (Schick and Schütze 2021a,b; Tam et al. 2021; Gao, Fisch, and Chen 2021; Schick and Schütze 2020; Liu et al. 2021a; Hu et al. 2021) transforms the input to a discrete textual string as a “prompt” to feed to a PLM.”
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022, background): “Although it lacks a rigorously theoretical proof, prompt learning inherits the few-shot property [Li and Liang, 2021; Schick and Schütze, 2020; Jin et al., 2021; Liu et al., 2021].”
The Art of Prompting: Event Detection based on Type Specific Prompts (2022, methodology): “It has shown competitive performance in a wide variety of applications in natural language processing (Raffel et al., 2020; Brown et al., 2020; Shin et al., 2020; Jiang et al., 2020; Lester et al., 2021; Schick and Schütze, 2021b).”
Social Simulacra: Creating Populated Prototypes for Social Computing Systems (2022, background): “Large language models can efectively perform a wide range of NLP tasks [5] such as language understanding [6, 32, 60, 78], text classifcation [36, 50, 86], and generation [59, 68] without the need for fne-tuning.”
Natural Instructions: Benchmarking Generalization to New Tasks from Natural Language Instructions (2021, background): “Another related line of work is the research on effective prompting of LMs (Shin et al., 2020; Schick and Schütze, 2020; Scao and Rush, 2021; Reynolds and McDonell, 2021; Liu et al., 2021; Tam et al., 2021).”