Large Language Models

Pretraining and foundation/decoder LLMs and model families (GPT, LLaMA, PaLM, Mistral, …).

A topic in the Science corpus, drawn from 1,003 papers.

Trusted voices in Large Language Models

Ordered by the trust Large Language Models's own network concentrates on each researcher, most first.

  1. Jacob Devlin — trusted by 1,913 researchers here
  2. Ming-Wei Chang — trusted by 1,893 researchers here
  3. Kristina Toutanova — trusted by 1,891 researchers here
  4. Kenton C. T. Lee — trusted by 1,891 researchers here
  5. Alec Radford — trusted by 1,584 researchers here
  6. Karthik Narasimhan — trusted by 946 researchers here
  7. Yoshua Bengio — trusted by 643 researchers here
  8. Kenneth Heafield — trusted by 362 researchers here
  9. Quoc V. Le — trusted by 476 researchers here
  10. Jeff Wu — trusted by 456 researchers here
  11. Noam Shazeer — trusted by 425 researchers here
  12. Dario Amodei — trusted by 414 researchers here

Most-cited papers in Large Language Models

Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.

  1. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019, North American Chapter of the Association for Computational Linguistics) — 118,136 citations
  2. Deep Learning (2016) — 78,707 citations
  3. Language Models are Few-Shot Learners (2020, Neural Information Processing Systems) — 61,005 citations
  4. RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019, arXiv.org) — 30,530 citations
  5. Language Models are Unsupervised Multitask Learners (2019) — 29,506 citations
  6. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019, Journal of machine learning research) — 26,606 citations
  7. GPT-4 Technical Report (2023) — 26,133 citations
  8. Training language models to follow instructions with human feedback (2022, Neural Information Processing Systems) — 22,660 citations
  9. LLaMA: Open and Efficient Foundation Language Models (2023, arXiv.org) — 20,887 citations
  10. Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, Neural Information Processing Systems) — 20,310 citations
  11. Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, arXiv.org) — 17,670 citations
  12. The Llama 3 Herd of Models (2024) — 17,085 citations
  13. Improving Language Understanding by Generative Pre-Training (2018) — 15,103 citations
  14. 5分で分かる!? 有名論文ナナメ読み:Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2020) — 14,205 citations
  15. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2019) — 13,006 citations
  16. Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, Neural Information Processing Systems) — 9,702 citations
  17. XLNet: Generalized Autoregressive Pretraining for Language Understanding (2019, Neural Information Processing Systems) — 9,461 citations
  18. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (2023) — 8,665 citations
  19. Scaling Laws for Neural Language Models (2020, arXiv.org) — 8,567 citations
  20. Machine learning: Trends, perspectives, and prospects (2015) — 8,448 citations
  21. PaLM: Scaling Language Modeling with Pathways (2022, Journal of machine learning research) — 8,214 citations
  22. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021) — 7,969 citations
  23. A Neural Probabilistic Language Model (2003) — 7,861 citations
  24. Robust Speech Recognition via Large-Scale Weak Supervision (2022, International Conference on Machine Learning) — 7,824 citations
  25. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024) — 7,648 citations
  26. BioBERT: a pre-trained biomedical language representation model for biomedical text mining (2019) — 7,641 citations
  27. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (2019) — 7,601 citations
  28. Large Language Models are Zero-Shot Reasoners (2022) — 7,591 citations
  29. Efficient Memory Management for Large Language Model Serving with PagedAttention (2023) — 7,142 citations
  30. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (2022) — 7,032 citations
  31. On the Opportunities and Risks of Foundation Models (2021) — 6,923 citations
  32. Flamingo: a Visual Language Model for Few-Shot Learning (2022, Neural Information Processing Systems) — 6,220 citations
  33. Recurrent neural network based language model (2010) — 6,125 citations
  34. The Power of Scale for Parameter-Efficient Prompt Tuning (2021) — 5,785 citations
  35. Longformer: The Long-Document Transformer (2020) — 5,720 citations
  36. Improved Baselines with Visual Instruction Tuning (2023) — 5,499 citations
  37. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning (2025, Nature) — 5,477 citations
  38. Finetuned Language Models Are Zero-Shot Learners (2021) — 5,221 citations
  39. Evolutionary-scale prediction of atomic level protein structure with a language model (2022) — 5,058 citations
  40. OPT: Open Pre-trained Transformer Language Models (2022) — 4,844 citations
  41. Toolformer: Language Models Can Teach Themselves to Use Tools (2023, Neural Information Processing Systems) — 4,753 citations
  42. A Survey of Large Language Models (2023) — 4,706 citations
  43. Large language models encode clinical knowledge (2022) — 4,703 citations
  44. ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks (2019) — 4,585 citations
  45. Tree of Thoughts: Deliberate Problem Solving with Large Language Models (2023) — 4,499 citations
  46. Transformer-XL: Attentive Language Models beyond a Fixed-Length Context (2019) — 4,418 citations
  47. Lost in the Middle: How Language Models Use Long Contexts (2023) — 4,355 citations
  48. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (2021, Journal of machine learning research) — 4,336 citations
  49. Sparks of Artificial General Intelligence: Early experiments with GPT-4 (2023) — 4,318 citations
  50. Scaling Instruction-Finetuned Language Models (2022, Journal of machine learning research) — 4,301 citations

Most active researchers

By papers in this topic: a volume measure, not trust.