LLaMA: Open and Efficient Foundation Language Models

By Hugo Touvron, Thibaut Lavril, Gautier Izacard, X. Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Naman Goyal, Eric Hambro and 5 more. arXiv.org, 2023

20,887 citations recorded by the source corpus.

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

Subfields: Artificial Intelligence, Large Language Models

View Paper: https://arxiv.org/abs/2302.13971

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Luke S. Zettlemoyer
  4. Percy Liang
  5. Yejin Choi
  6. Graham Neubig
  7. Yoshua Bengio
  8. Quoc V. Le

In Large Language Models

  1. Luke S. Zettlemoyer
  2. Yejin Choi
  3. Furu Wei
  4. Noah A. Smith
  5. Jianfeng Gao
  6. Graham Neubig
  7. Hinrich Schütze
  8. Hannaneh Hajishirzi

Citations in the corpus