An Explanation of In-context Learning as Implicit Bayesian Inference
By Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma. 2021
1,108 citations recorded by the source corpus.
Large language models (LMs) such as GPT-3 have the surprising ability to do in-context learning, where the model learns to do a downstream task simply by conditioning on a prompt consisting of input-output examples. The LM learns from these examples without being explicitly pretrained to learn. Thus, it is unclear what enables in-context learning. In this paper, we study how in-context learning can emerge when pretraining documents have long-range coherence. Here, the LM must infer a latent document-level concept to generate coherent next tokens during pretraining. At test time, in-context learning occurs when the LM also infers a shared latent concept between examples in a prompt. We prove when this occurs despite a distribution mismatch between prompts and pretraining data in a setting where the pretraining distribution is a mixture of HMMs. In contrast to messy large-scale datasets used to train LMs capable of in-context learning, we generate a small-scale synthetic dataset (GINC) where Transformers and LSTMs both exhibit in-context learning. Beyond the theory, experiments on GINC exhibit large-scale real-world phenomena including improved in-context performance with model scaling (despite the same pretraining loss), sensitivity to example order, and instances where zero-shot is better than few-shot in-context learning.
Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Jianfeng Gao
- Yoshua Bengio
- Quoc V. Le
- Yejin Choi
- Graham Neubig
In Large Language Models
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Wenhu Chen
- Percy Liang
- Antonia Creswell
- Ming Zhou
- Denny Zhou
- Jonathan Berant
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Denny Zhou
- Wenhu Chen
- Hannaneh Hajishirzi
- Antonia Creswell
- Jonathan Berant
- Jason Wei
- Xinyun Chen
Citations in the corpus
- Can language models learn from explanations in context? (2022, background): “Recent observations that LMs can perform tasks from a few examples (Brown et al., 2020; Rad-ford et al., 2019) have stimulated discussion about what underlies that learning—e.g., Xie et al. (2021) suggest that in-context learning is a form of implicit Bayesian inference.”
- STaR: Bootstrapping Reasoning With Reasoning (2022, methodology): “Some have studied in-context learning based on the language modeling objective in terms of Bayesian inference [13] while others have attempted to describe the process more mechanistically in terms of “induction heads” [14].”
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (2022, background): “Xie et al. (2022) provide theoretical analysis that in-context learning can be formalized as Bayesian inference that uses the demonstrations to recover latent concepts.”
- Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “Zero-shot and few-shot prompting recently have been shown to be highly effective for transfer learning (Brown et al., 2020; Xie et al., 2021; Min et al., 2022).”
- Learning To Retrieve Prompts for In-Context Learning (2021, background): “Xie et al. (2021) suggests that in-context learning occurs when the model infers a shared latent concept between examples in a prompt.”
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022, methodology): “Recently, there have been several works (Xie et al., 2021; Min et al., 2022) aiming to understand how and why in-context learning works.”
- A Survey of Large Language Models (2023, positive): “Recent studies either conduct extensive experiments for investigating the effect of emergent abilities and the contributing factors to such abilities [220, 237, 421], or explain some specific abilities with existing theoretical frameworks [60, 231].”
- Transformers learn to implement preconditioned gradient descent for in-context learning (2023, background): “…property has become the focus of body of recent research that aims to shed light on the underlying mechanism of large language models (Garg et al., 2022; Akyürek et al., 2022; von Oswald et al., 2023; Li and Malik, 2017; Min et al., 2021; Xie et al., 2021; Elhage et al., 2021; Olsson et al., 2022).”
- Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time (2023, methodology): “Inspired by the connections between LLMs, Hidden Markov Models (Xie et al., 2022; Baum & Petrie, 1966), and the classic Viterbi algorithm (Viterbi, 1967), we hypothesize that for pre-trained LLMs, contextual sparsity exists given any input.”
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face (2023, background): “The powerful capability of LLMs also drives many emergent research topics (e.g., in-context learning [1, 7, 8], instruction learning [9, 10, 11, 12, 13, 14], and chain-of-thought prompting [15, 16, 17, 18]) to further investigate the huge potential of LLMs, and brings unlimited possibilities for us…”
- MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning (2023, methodology): “Self-critic (Gou et al., 2023) and self-eval (Xie et al., 2022) both adopt self-evaluation to enhance the robustness of the generated program.”
- Larger language models do in-context learning differently (2023, background): “On a theoretical level, Xie et al. (2022) provide an explanation of in-context learning in which transformers infer tasks from exemplars because they are trained to infer latent concepts during pretraining, and prior knowledge obtained from pretraining data can then be applied to incontext examples.”
- On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model (2022, background): “On the other hand, another group of papers tries to explain the mechanism of in-context few-shot learning (Min et al., 2022; Xie et al., 2022).”
- A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, background): “Currently, the application of prompt-based methods and their variants (e.g., in-context learning [148], instruction tuning [94], etc.) in large-scale language models is a hot academic topic.”
- What Can Transformers Learn In-Context? A Case Study of Simple Function Classes (2022, background): “Xie et al. [74] propose a Bayesian inference framework explaining how in-context learning works despite formatting differences between training and inference distributions.”
- Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations (2022, background): “, 2021; Reynolds and McDonell, 2021), and (2) what main factors exist behind its success (Min et al., 2022; Shin et al., 2022; Xie et al., 2022).”