WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions
By Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Daxin Jiang. 2023
1,266 citations recorded by the source corpus.
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Human evaluations on a complexity-balanced test bed and Vicuna's testset show that instructions from Evol-Instruct are superior to human-created ones. By analyzing the human evaluation results of the high complexity part, we demonstrate that outputs from our WizardLM are preferred to outputs from OpenAI ChatGPT. In GPT-4 automatic evaluation, WizardLM achieves more than 90\% capacity of ChatGPT on 17 out of 29 skills. Even though WizardLM still lags behind ChatGPT in some aspects, our findings suggest that fine-tuning with AI-evolved instructions is a promising direction for enhancing LLMs. Our code and data are public at https://github.com/nlpxucan/WizardLM
Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Yejin Choi
- Percy Liang
- Graham Neubig
- Zhiyuan Liu
- Quoc V. Le
In Large Language Models
In Prompting & Reasoning
Citations in the corpus
- Judging LLM-as-a-judge with MT-Bench and Chatbot Arena (2023, background): “There has been a proliferation of LLM-based chat assistants (chatbots) that leverage supervised instruction fine-tuning and reinforcement learning with human feedback (RLHF) to unlock new instruction following and conversational abilities [30, 2, 29, 8, 51, 47, 14].”
- LongForm: Effective Instruction Tuning with Reverse Instructions (2023, neutral): “Thus, our dataset features a diverse collection of instructions (including human-written, template-based and LLM-generated), each paired with a human-written output.”
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, neutral): “…Type: Text Datasets UltraFeedback (en) [48], UltraChat (en) [58], Unnatural-Instructions (en) [90], NoRobots (en) [196], MOSS (en) [221], LIMA (en) [314], SlimOrca (en) [142], WizardLM-Evol-Instruct-70K (en) [265], Llama-3-Magpie-Pro (en) [266], Magpie-Qwen2-Pro (en & zh) [266], KOpen-HQ-Hermes-2.”
- Baichuan 2: Open Large-scale Language Models (2023, background): “Those chat models have demonstrated a marked improvement in understanding human instructions and solving complex tasks (Chiang et al., 2023; Xu et al., 2023; Sun et al., 2023).”
- Rethinking Benchmark and Contamination for Language Models with Rephrased Samples (2023, background): “Moreover, there has been a growing interest in training models using synthetic data produced by LLMs (e.g., GPT-4) (Gunasekar et al., 2023; Taori et al., 2023; Wang et al., 2023b; Xu et al., 2023; Mukherjee et al., 2023), in which contamination may be even harder to detect by string matching.”
- InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents (2024, background): “Figure 4 shows the valid rates of different agents under base attacks and the Arena Elo ratings of LLMs (Zheng et al., 2023), which are indicative of their overall capabilities.”
- Character-LLM: A Trainable Agent for Role-Playing (2023, methodology): “…(Bai et al., 2022) methods dominate the aligning of LLMs. Meth-ods such as Alpaca, and Vicuna (Taori et al., 2023; Chiang et al., 2023) show the possibility of simply fine-tuning LLMs to align them to certain applications with simple self-generated instructions (Wang et al., 2022; Xu et al., 2023).”
- DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrapping (2023, background): “Instruction-Tuning Data Collection The collection of high-quality instruction-tuning data (Xu et al., 2023; Yin et al., 2023; Honovich et al., 2023) is a pressing issue in enhancing the capability of instruction-following.”
- Mixture-of-Agents Enhances Large Language Model Capabilities (2024, background): “For instance, some models excel at complex instruction following (Xu et al., 2023a) while others may be better suited for code generation (Roziere et al., 2023; Guo et al., 2024).”
- Kimi K2: Open Agentic Intelligence (2025, positive): “Second, we systematically evolve [83] synthetic tools through a hierarchical domain generation process: we begin with key categories (e.g., financial trading, software applications, robot control), then evolve multiple specific application domains within each category.”
- Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model (2024, neutral): “Evolutionary search has been adopted in both code generation (Liventsev et al., 2023; Ma et al., 2023; Lehman et al., 2024; Hemberg et al., 2024) and text generation (Guo et al., 2023b; Fernando et al., 2023; Xu et al., 2023a).”
- Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing (2024, methodology): “In contrast, the second type of method uses LLMs to produce synthetic instructions [16, 31, 46, 47, 53, 55, 58, 59].”
- Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023, background): “You take one cup from the red bucket and pour it into the blue bucket.”
- Yi: Open Foundation Models by 01.AI (2024, methodology): “We use the following techniques to improve prompt distribution selection, response formatting, and chain-of-thought formatting: (1). for prompt distribution selection, drawing inspiration from WizardLM[83], we develope compound instructions and progressively evolved them to increase their complexity”
- How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2023, background): “, 2023a), prompting proprietary LLM (Wang et al., 2023c; Taori et al., 2023; Lei et al., 2023; Xu et al., 2023).”