Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

By Ruiqi Zhong, Kristy Lee, Zheng Zhang, Daniel Klein. 2021

188 citations recorded by the source corpus.

Large pre-trained language models (LMs) such as GPT-3 have acquired a surprising ability to perform zero-shot learning. For example, to classify sentiment without any training examples, we can"prompt"the LM with the review and the label description"Does the user like this movie?", and ask whether the next word is"yes"or"no". However, the next word prediction training objective is still misaligned with the target zero-shot learning objective. To address this weakness, we propose meta-tuning, which directly optimizes the zero-shot learning objective by fine-tuning pre-trained language models on a collection of datasets. We focus on classification tasks, and construct the meta-dataset by aggregating 43 existing datasets and annotating 441 label descriptions in a question-answering (QA) format. When evaluated on unseen tasks, meta-tuned models outperform a same-sized QA model and the previous SOTA zero-shot learning system based on natural language inference. Additionally, increasing parameter count from 220M to 770M improves AUC-ROC scores by 6.3%, and we forecast that even larger models would perform better. Therefore, measuring zero-shot learning performance on language models out-of-the-box might underestimate their true potential, and community-wide efforts on aggregating datasets and unifying their formats can help build models that answer prompts better.

Subfields: Artificial Intelligence, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Prompting & Reasoning

Citations in the corpus

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, background): “…wide range of tasks using human annotated instructions and feedback (Ouyang et al., 2022), whereas the other focusing on adding instructions via annotations or automatically on publicly available benchmarks and datasets (Wang et al., 2022; Wei et al., 2022a; Sanh et al., 2022; Zhong et al., 2021).”
GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, background): “There is a significant and growing body of work on the influence of the syntactic structure of finetuning data on downstream performance (Zhong et al., 2021; Tan et al., 2021; Sanh et al., 2021; Wei et al., 2021).”
Scaling Instruction-Finetuned Language Models (2022, background): “Finally, whereas much of prior work ﬁnetunes on either zero-shot without exemplars (Zhong et al., 2021; Wei et al., 2021; Sanh et al., 2021; Wang et al., 2022a) or few-shot with exemplars (Ye et al., 2021; Wei et al., 2021; Min et al., 2022; Wang et al., 2022c), our process ﬁnetunes on a mixture of…”
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, neutral): “Similar to ours, recent works study the impact of language instructions in modifying a model behavior (Hase and Bansal, 2021; Ye and Ren, 2021; Gupta et al., 2021; Zhong et al., 2021).”
Learning by Distilling Context (2022, background): “Many recent works show that language models can learn from abstract task deﬁnitions (Zhong et al., 2021; Mishra et al., 2022; Wei et al., 2022a; Sanh et al., 2022), natural language explanations (Scheurer et al., 2022; Wang et al., 2022b), and concrete in-context examples (Min et al., 2022; Chen et…”
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, background): “The current mainstream of zero-shot learning is based on transfer learning: By converting a set of tasks with abundant annotations into instruction templates [42, 54, 70, 74], entailment pairs [79, 80] or question-answer formats [50, 86] and fine-tuning PLMs on them, the PLMs acquire the cross-task ”
QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “Approaches and datasets include MetaICL [40], MetaTuning [74], InstructGPT [43], FLAN [63, 12], PromptSource [3], Super-NaturalInstructions [62, 51], Self-instruct [60], UnnaturalInstructions [26], OPT-IML [29], UnifiedSKG[68], OIG/Chip2 [32], Alpaca [56], Vicuna [10], Koala [20], and…”
Instruction Tuning with GPT-4 (2023, background): “Instruction tuning of LLMs is an increasingly popular research direction in NLP (Zhong et al., 2021; Ouyang et al., 2022; Wei et al., 2021).”
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022, background): “N AT I NST .”
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, background): “, 2020), which can be further improved by multi-task learning (Zhong et al., 2021), e.”
MetaICL: Learning to Learn In Context (2021, methodology): “The number of tasks is the largest among recent related work: we have 142 unique tasks, while Khashabi et al. (2020), Zhong et al. (2021), Mishra et al. (2022), Wei et al. (2022) and Sanh et al. (2022) use 32, 62, 61, 42 and 62 tasks, respectively.”
ZeroPrompt: Scaling Prompt-Based Pretraining to 1, 000 Tasks Improves Zero-Shot Generalization (2022, methodology): “Further, other works proposed to include a set of supervised tasks into pretraining (Zhong et al., 2021; Wei et al., 2021; Sanh et al., 2021), and prompts are often used in the framework to unify the tasks.”
Poisoning Language Models During Instruction Tuning (2023, methodology): “Instruction-tuned Language Models The de-facto standard method for building state-of-the-art large LMs is via “instruction” or “meta” ﬁnetuning (Ouyang et al., 2022; Chung et al., 2022; Min et al., 2022; Zhong et al., 2021, inter alia ).”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, background): “This setting is attracting attention in NLP community rapidly and is also explored in very recent work (Zhong et al., 2021; Mishra et al., 2021; Bragg et al., 2021; Wei et al., 2021).”
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right (2021, background): “Recent work suggests it may be possible to finetune language models in order to improve their zeroshot and few-shot capabilities on a large swathe of tasks (Wei et al., 2021; Zhong et al., 2021).”
FLEX: Unifying Evaluation for Few-Shot NLP (2021, methodology): “[80] also show the benefit of performing meta-tuning on a variety of datasets; while their task setup as Q/A is similar to UniFew, they focus exclusively on binary zero-shot classification tasks and, unlike UniFew, do not handle multi-class or few-shot problems.”
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2021, background): “To overcome the shortcomings of discrete spaces, Li and Liang (2021); Liu et al. (2021); Han et al. (2021b); Hambardzumyan et al. (2021); Zhong et al. (2021b) explore to combine hard prompts and soft prompts.”
Meta-learning via Language Model In-context Tuning (2021, methodology): “BinaryClfs does not come with an ofﬁcial validation set.”
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022, background): “Going beyond multi-task training, these approaches show better generalization to unseen tasks when prompted with a few examples (Bragg et al., 2021; Min et al., 2022a,b) or language definitions and constraints (Weller et al., 2020; Zhong et al., 2021b; Xu et al., 2022).”