Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

By Laria Reynolds, Kyle McDonell. 2021

1,354 citations recorded by the source corpus.

Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models’ novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. We discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions and techniques for encouraging deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, we also introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. Finally, we discuss how these more general methods of interacting with language models can be incorporated into existing and future benchmarks and practical applications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Percy Liang
  4. Yejin Choi
  5. Jianfeng Gao
  6. Graham Neubig
  7. Quoc V. Le
  8. Noah A. Smith

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Bernt Schiele

In Large Language Models

  1. Luke S. Zettlemoyer
  2. Yejin Choi
  3. Hinrich Schütze
  4. Hannaneh Hajishirzi
  5. Yi Tay
  6. Noah A. Smith
  7. Graham Neubig
  8. Percy Liang

In Prompting & Reasoning

  1. Denny Zhou
  2. Luke S. Zettlemoyer
  3. Swaroop Mishra
  4. Jason Wei
  5. Yejin Choi
  6. Xinyun Chen
  7. Hannaneh Hajishirzi
  8. Chitta Baral

Citations in the corpus