The Llama 3 Herd of Models

By Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, A. Schelten, Amy Yang, Angela Fan and 517 more. 2024

17,085 citations recorded by the source corpus.

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Multimodal Learning

View Paper: https://arxiv.org/abs/2407.21783

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Luke S. Zettlemoyer
  4. Yoshua Bengio
  5. Percy Liang
  6. Yejin Choi
  7. Quoc V. Le
  8. Graham Neubig

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Cordelia Schmid

In Large Language Models

  1. Jason Aaron Edward Weston
  2. Graham Neubig
  3. Noah A. Smith
  4. Kai-Wei Chang
  5. Wenhu Chen
  6. Armand Joulin
  7. Huan Sun
  8. Oriol Vinyals

In Multimodal Learning

  1. Aniruddha Kembhavi
  2. Zhe Gan
  3. Jiasen Lu
  4. Jason Aaron Edward Weston
  5. Antoine Miech
  6. Devi Parikh
  7. Jean-Baptiste Alayrac
  8. Jingren Zhou

Citations in the corpus