OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

By S. Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura and 8 more. 2022

307 citations recorded by the source corpus.

Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding of the performance trade-offs of different decisions made during the instruction-tuning process. These decisions include the scale and diversity of the instruction-tuning benchmark, different task sampling strategies, fine-tuning with and without demonstrations, training using specialized datasets for reasoning and dialogue, and finally, the fine-tuning objectives themselves. In this paper, we characterize the effect of instruction-tuning decisions on downstream task performance when scaling both model and benchmark sizes. To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks. Through the lens of this framework, we first present insights about instruction-tuning decisions as applied to OPT-30B and further exploit these insights to train OPT-IML 30B and 175B, which are instruction-tuned versions of OPT. OPT-IML demonstrates all three generalization abilities at both scales on four different evaluation benchmarks with diverse tasks and input formats -- PromptSource, FLAN, Super-NaturalInstructions, and UnifiedSKG. Not only does it significantly outperform OPT on all benchmarks but is also highly competitive with existing models fine-tuned on each specific benchmark. We release OPT-IML at both scales, together with the OPT-IML Bench evaluation framework.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

LLaMA: Open and Efficient Foundation Language Models (2023, methodology): “LLaMA-I (65B) outperforms on MMLU existing instruction finetuned models of moderate sizes, but are still far from the state-of-the-art, that is 77.4 for GPT code-davinci-002 on MMLU (numbers taken from Iyer et al. (2022)).”
Augmented Language Models: a Survey (2023, background): “Further, several instruction fine-tuning approaches (Ouyang et al., 2022; Chung et al., 2022; Iyer et al., 2022; Ho et al., 2022) use chain-of-thought style prompts to achieve remarkable improvements on popular benchmarks such as BBH (Srivastava et al.”
QLoRA: Efficient Finetuning of Quantized LLMs (2023, background): “The authors show that models trained on this collection outperform other public collections including the original FLAN 2021 [63], T0++ [51], Super-Natural Instructions [61], and OPT-IML [29].”
Visual Instruction Tuning (2023, neutral): “…OPT [60] to follow natural language instructions and complete real-world tasks, researchers have explored methods for LLM instruction-tuning [37, 52, 51], leading to instruction-tuned counterparts such as InstructGPT [37]/ChatGPT [35], FLAN-T5 [11], FLAN-PaLM [11], and OPT-IML [22], respectively.”
Instruction Tuning with GPT-4 (2023, methodology): “To evaluate data quality, we train a reward model based on OPT 1.3B (Iyer et al., 2022) to rate different responses.”
TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation (2023, methodology): “Considering that instruction tuning is core to letting the LLM learn to solve different tasks and have strong generalization ability [22, 23, 37], we propose a lightweight tuning framework to adapt LLMs for recommendations, named TALLRec.”
A Survey on Large Language Models for Code Generation (2024, unknown): “For example, Google revealed that the fine-tuning of sentiment analysis boosted the accuracy of LLMs by 10% [39].”
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model (2024, methodology): “Here, we are interested in pre-training efforts which are massively multilingual (Xue et al., 2020; Chung et al., 2023; Shliazhko et al., 2022; Scao et al., 2022; Lin et al., 2022; Devlin et al., 2019; Conneau et al., 2019; Khanuja et al., 2021; Oladipo et al., 2023; Alabi et al., 2022).”
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources (2023, methodology): “Prior work has examined models trained on varying dataset sources with the aim of identifying ‘the best mixture’ [31, 24], but is often limited to examining only benchmark performance, and covers a smaller number of instruction sources than in this work.”
Specializing Smaller Language Models towards Multi-Step Reasoning (2023, methodology): “Currently, the new recipe for training LLMs is to ﬁrst train a base model (e.g., GPT-3, PaLM, OPT), then elicit the abilities of the base model by instruction tuning (e.g., GPT-3 → InstructGPT Ouyang et al., 2022; PaLM → FlanPaLM Chung et al., 2022, OPT → OPT-IML Iyer et al., 2022, also see Fig.”
Tool Learning with Foundation Models (2023, unknown): “Promisingly, such generalization ability can further be enhanced by scaling up both the model size and the quantity or diversity of training instructions (Iyer et al., 2022).”
CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society (2023, background): “Hand-crafted instruction instances are available in [71], while leveraging previously crowdsourced NLP datasets is a less labor-intensive curation approach [72, 41, 46, 32].”
Large Language Models: A Survey (2024, unknown): “…(or techniques used for efficient developments of LLMs) includes Inner-Monologue [100], Megatron-Turing NLG [101], LongFormer [102], OPT-IML [103], MeTaLM [104], Dromedary [105], Palmyra [106], Camel [107], Yalm [108], MPT [109], ORCA-2 [110], Gorilla [67], PAL [111], Claude [112], CodeGen…”
A Comprehensive Overview of Large Language Models (2023, unknown): “Generating reasons is possible only by using different prompting styles, whereas to improve LLMs further on reasoning tasks many methods [25], [24] train them on reasoning datasets.”
Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models (2023, unknown): “With the scaling law of instruction data that more task types and data can improve the performance of instruction tuning (Xu et al., 2022), subsequent works collect more data from existing datasets to expand tasks to the thousands, e.g., Ze-roPrompt (Xu et al., 2022), Super-Natural Instructions (Wan”
Gorilla: Large Language Model Connected with Massive APIs (2023, unknown): “Besides prompting, there have also been efforts to pretrain language models specifically for code generation [25, 21, 26].”
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction (2023, background): “Notably, InstructGPT [2], FLAN-T5 [31], OPT-IML [33] demonstrated remarkable performance on specific tasks after being fine-tuned with instruction data.”