Scaling Instruction-Finetuned Language Models
By Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma and 22 more. Journal of machine learning research, 2022
4,301 citations recorded by the source corpus.
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.
Subfields: Artificial Intelligence, Large Language Models, Alignment & RLHF
View Paper: https://arxiv.org/abs/2210.11416
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- Quoc V. Le
- Yejin Choi
- Noah A. Smith
In Large Language Models
- Luke S. Zettlemoyer
- Hinrich Schütze
- Yejin Choi
- Timo Schick
- Jianfeng Gao
- Hannaneh Hajishirzi
- Noah A. Smith
- Graham Neubig
In Alignment & RLHF
- Luke S. Zettlemoyer
- Jianfeng Gao
- Ari Holtzman
- Tim Dettmers
- Artidoro Pagnoni
- Yejin Choi
- Xinyun Chen
- Mari Ostendorf
Citations in the corpus
- OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “Instruction fine-tuning is shown (Wei et al., 2022a; Sanh et al., 2022; Chung et al., 2022a) to significantly improve the zero- and few-shot performance of large pretrained LMs (LLM).”
- Emergent Abilities of Large Language Models (2022, positive): “By finetuning on a mixture of tasks phrased as instructions, language models have been shown to respond appropriately to instructions describing an unseen task (Ouyang et al., 2022; Wei et al., 2022a; Sanh et al., 2022; Chung et al., 2022).”
- The False Promise of Imitating Proprietary LLMs (2023, methodology): “At the same time, open-source LMs are becoming increasingly accurate, with models like LLaMA and FLAN-T5 providing many of the same basic capabilities as their commercial counterparts, albeit at a lower level of performance (Touvron et al., 2023; Chung et al., 2022).”
- AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models (2023, neutral): “Large Foundation Model: Recently, large foundation models, like LLMs (e.g., GPT-3 (Brown et al., 2020), GPT-4 (OpenAI, 2023), OPT (Zhang et al., 2022a) and FLAN-T5 (Chung et al., 2022)) have successfully demonstrated unprecedented performance in a wide range of natural language tasks.”
- Augmented Language Models: a Survey (2023, methodology): “Typically, supervised learning is done after an initial pre-training with a language modeling objective (Ouyang et al., 2022; Chung et al., 2022); an exception to this is recent work by Taylor et al.”
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023, methodology): “Where Chung et al. (2022) focuses on the emergent and state-of-the-art results of combining Flan 2022 with PaLM 540B, this work focuses in on the details of the instruction tuning methods themselves, ablating individual factors, and comparing them directly to prior work by keeping the pretrained…”
- Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, methodology): “Additionally, they show a direct correlation between the size and diversity of the “instructional” data and the generalizability of resulting models to unseen tasks (Wang et al., 2022; Chung et al., 2022).”
- Aligning Language Models with Preferences through f-divergence Minimization (2023, background): “, 2021), supervised fine-tuning on curated data (Solaiman & Dennison, 2021; Ngo et al., 2021; Welbl et al., 2021; Chung et al., 2022) or selected samples from the model (Zelikman et al.”
- UL2: Unifying Language Learning Paradigms (2022, methodology): “We pretty much use the same settings and Flan mixture as the Flan2 paper (Chung et al., 2022).”
- QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “Approaches and datasets include MetaICL [40], MetaTuning [74], InstructGPT [43], FLAN [63, 12], PromptSource [3], Super-NaturalInstructions [62, 51], Self-instruct [60], UnnaturalInstructions [26], OPT-IML [29], UnifiedSKG[68], OIG/Chip2 [32], Alpaca [56], Vicuna [10], Koala [20], and…”
- Visual Instruction Tuning (2023, background): “In the natural language processing (NLP) community, to enable LLMs such as GPT-3 [7], T5 [41], PaLM [10], and OPT [60] to follow natural language instructions and complete real-world tasks, researchers have explored methods for LLM instruction-tuning [37, 52, 51], leading to instruction-tuned counte”
- Instruction Tuning with GPT-4 (2023, methodology): “For example, scaling instruction-finetuned language models (Chung et al., 2022) is built on top of FLAN (Wei et al.”
- GPTScore: Evaluate as You Desire (2023, methodology): “, 2022b), FLAN (Chung et al., 2022), and GPT3 (instruction-based (Ouyang et al.”
- Improved Baselines with Visual Instruction Tuning (2023, background): “However, in NLP, the FLAN family [12, 50] shows that adding a large number of academic language tasks for instruction tuning can effectively improve the gen-26297 eralization ability.”
- LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (2023, methodology): “In this section, we compare the generation quality of LLaMA-Adapter with LLaMA-I [61], an instruction-fine-tuned LLaMA 65B model following [11].”