Instruction Tuning for Large Language Models: A Survey

By Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu and 1 more. ACM Computing Surveys, 2023

905 citations recorded by the source corpus.

This article surveys research works in the quickly advancing field of instruction tuning (IT), a crucial technique to enhance the capabilities and controllability of large language models (LLMs). Instruction tuning refers to the process of further training LLMs on a dataset consisting of (instruction, output) pairs in a supervised fashion, which bridges the gap between the next-word prediction objective of LLMs and the users’ objective of having LLMs adhere to human instructions. In this work, we make a systematic review of the literature, including the general methodology of IT, the construction of IT datasets, the training of IT models, and applications to different modalities, domains and application, along with analysis of aspects that influence the outcome of IT (e.g., generation of instruction outputs, size of the instruction dataset). We also review the potential pitfalls of IT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies and suggest some avenues for fruitful research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f0950a3f27c0fefffba60ae1c9a8ee360d5eb55f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities (2024, methodology): “Fine-tuning an LLM model with 7 billion parameters, such as GPT-4-LLM [94], could take nearly 3 hours on an 8 × 80GB A100 machine, which is extremely time-consuming [95].”
StruQ: Defending Against Prompt Injection with Structured Queries (2024, neutral): “…trained in at least two stages: (a) a base LLM is trained for text completion (next-word prediction), (b) then the base LLM is fine-tuned to understand and act on instructions (using instruction tuning), adhere to safety guidelines, or engage in extended dialogue sequences [20, 21, 22, 23].”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “Considering that LLMs have inherent capability boundaries established during pre-training, SFT utilizes high-quality instructions along with their corresponding responses to empower LLMs to follow user instructions, unlocking their acquired abilities in this process.”
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model (2024, background): “…pairs of prompts and completions , and has been shown to significantly improve the helpfulness and general instruction following capabilities of large language models (LLMs) (Sanh et al., 2022; Wei et al., 2021; Iyer et al., 2022; Muennighoff et al., 2023c; Chung et al., 2022; Zhang et al., 2023c).”
Large Language Models: A Survey (2024, neutral): “This is the so-called instruction tuning [133].”
Hallucination of Multimodal Large Language Models: A Survey (2024, neutral): “To address this disparity, the concept of Supervised Fine-Tuning (SFT) [125] has been introduced.”
A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, neutral): “Since memory is important for humans, designing memory modules is also significant for the agents.”
A New Pipeline for Generating Instruction Dataset via RAG and Self Fine-Tuning (2024, unknown): “The success of ChatGPT has popularized the use of Instruction Tuning [6], subsequently validated by related research as the most effective fine-tuning approach for question-answering tasks [7].”
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation (2024, unknown): “This knowledge, embedded in the model weights, can be accessed through model prompting after an alignment step [65,92], transforming LLMs into universal Question Answering (QA) tools and sparking an unprecedented surge in commercial and scientific interest.”
Query-efficient model evaluation using cached responses (2026, unknown): “These trends motivate the need for query-efficient alternatives to full benchmark evaluation.”