Llama 2: Open Foundation and Fine-Tuned Chat Models
By Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko-lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale and 64 more. arXiv.org, 2023
17,670 citations recorded by the source corpus.
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.
Subfields: Artificial Intelligence, Large Language Models, Alignment & RLHF
View Paper: https://arxiv.org/abs/2307.09288
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Yejin Choi
- Yoshua Bengio
- Quoc V. Le
- Graham Neubig
In Large Language Models
- Luke S. Zettlemoyer
- Yejin Choi
- Noah A. Smith
- Furu Wei
- Jianfeng Gao
- Hannaneh Hajishirzi
- Hinrich Schütze
- Graham Neubig
In Alignment & RLHF
- Jianfeng Gao
- Yejin Choi
- Rafael Rafailov
- Dylan Hadfield-Menell
- Xinyun Chen
- Mari Ostendorf
- Noah A. Smith
- David Krueger
Citations in the corpus
- Mistral 7B (2023, methodology): “Mistral 7B outperforms the previous best 13B model (Llama 2, [26]) across all tested benchmarks, and surpasses the best 34B model (LLaMa 34B, [25]) in mathematics and code generation.”
- Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “More recently, red team-ing is also a crucial step for large language models (Bai et al., 2022a; Touvron et al., 2023).”
- Llemma: An Open Language Model For Mathematics (2023, background): “, 2022), and algorithmic reasoning (Zhou et al., 2022; Zhang et al., 2023).”
- Improved Baselines with Visual Instruction Tuning (2023, background): “In NLP, findings [48] suggest that the capability of the base LLM can affect its instruction-tuned successors.”
- Choose Your Weapon: Survival Strategies for Depressed AI Academics (2023, background): “Having access to pretrained large models like ViT in vision [4] or the Llama family for text [25] saves you time and effort as you can simply resue them, and fine-tune them for your own specific problem.”
- From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline (2024, methodology): “To see whether “quality score” assigned during BenchBuilder’s pipeline correlates with separability and agreement, we sample 50 prompts per score and compare the responses from GPT-4 and Llama-2-70b-Chat [43], with GPT-4-Turbo as judge.”
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark (2024, methodology): “We employ eight models: Llama-2-7B, Llama-2-7B-Chat, Llama-2-13B, Llama-2-13B-Chat [35], Mistral-7B [20], Gemma-7B [34], Yi-6B, and Yi-6B-Chat [43].”
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024, neutral): “…a process-supervised reward model, (6) the WizardMath series (Luo et al., 2023) which improves mathematical reasoning in Mistral 7B and Llama-2 70B (Touvron et al., 2023) As shown in Table 5, under the evaluation setting where tool use is disallowed, DeepSeekMath-Instruct 7B demonstrates strong…”
- Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing (2024, methodology): “We follow [50] and use a cosine learning rate schedule with an initial learning rate of 2 × 10 − 5 when fine-tuning Llama-3 and Qwen1.”
- Steering Llama 2 via Contrastive Activation Addition (2023, methodology): “We apply the technique to Llama 2, a collection of pretrained and finetuned LLMs ranging in scale from 7 to 70 billion parameters (Touvron et al., 2023), primarily focusing on Llama 2 Chat, which is optimized for dialogue use-cases and finetuned using RLHF for safety.”
- Large Language Models Are Not Robust Multiple Choice Selectors (2023, methodology): “We experiment with 20 LLMs from popular LLM families across various sizes : llama-7/13/30/65B (Touvron et al., 2023a), llama-2(-chat)-7/13/70B (Touvron et al., 2023b), vicuna-v1.”
- VILA: On Pre-training for Visual Language Models (2023, positive): “There is a growing effort from the open-source community to build a strong base LLM [55, 56, 65], the conversational variants [14, 54] and the parameter-efficient finetuned versions of large LLMs [40, 64].”
- The Effect of Sampling Temperature on Problem Solving in Large Language Models (2024, background): “Despite significant interest in LLMs and progress in LLM best practices, many open questions remain about optimal prompt-engineering techniques and inference hyperparameters for LLMs.”
- Safe RLHF: Safe Reinforcement Learning from Human Feedback (2023, neutral): “Ultimately, our research was conducted using data from Llama-1 (Touvron et al., 2023a) and Alpaca (Taori et al., 2023) models which were considering predate Llama-2 (Touvron et al., 2023b).”
- LiveBench: A Challenging, Contamination-Free LLM Benchmark (2024, neutral): “For small open-source models, we include llama-2-7b-chat-hf [59], llama-3-8b-instruct [39], mistral-7b-instruct-v0.”