Instruction Tuning with GPT-4

By Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao. 2023

837 citations recorded by the source corpus.

Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Alignment & RLHF

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Mohit Bansal
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Yejin Choi
  6. Yoshua Bengio
  7. Noah A. Smith
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Xiaogang Wang
  4. Trevor Darrell
  5. Ming-Hsuan Yang
  6. Yu Qiao
  7. Andrew Zisserman
  8. Jitendra Malik

In Large Language Models

  1. Luke S. Zettlemoyer
  2. Hannaneh Hajishirzi
  3. Noah A. Smith
  4. Percy Liang
  5. Danqi Chen
  6. Jianfeng Gao
  7. Nan Duan
  8. Yu Meng

In Alignment & RLHF

  1. Tianyi Zhang
  2. Noah A. Smith
  3. Ari Holtzman
  4. Yu Meng
  5. Percy Liang
  6. Pieter Abbeel
  7. Carlos Guestrin
  8. Sergey Levine

Citations in the corpus