Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

By Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei. Annual Meeting of the Association for Computational Linguistics, 2023

401 citations recorded by the source corpus.

Large pretrained language models have shown surprising In-Context Learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without additional parameter updates. Despite the great success in performance, the working mechanism of ICL still remains an open problem. In order to better understand how ICL works, this paper explains language models as meta-optimizers and understands ICL as a kind of implicit ﬁnetuning. Theoretically, we ﬁgure out that the Transformer attention has a dual form of gradient descent based optimization. On top of it, we understand ICL as follows: GPT ﬁrst produces meta-gradients according to the demonstration examples, and then these meta-gradients are applied to the original GPT to build an ICL model. Experimentally, we comprehensively compare the behavior of ICL and explicit ﬁnetuning based on real tasks to provide empirical evidence that supports our understanding. The results prove that ICL behaves similarly to explicit ﬁnetuning at the prediction level, the representation level, and the attention behavior level. Further, inspired by our understanding of meta-optimization, we design a momentum-based attention by analogy with the momentum-based gradient descent algorithm. Its consistently better performance over vanilla attention supports our understanding again from another aspect, and more impor-tantly, it shows the potential to utilize our understanding for future model designing.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d86ca0894cb4d165eb5ef45b73526ca8b4cdd725

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Larger language models do in-context learning differently (2023, background): “…that models primarily rely on semantic prior knowledge), other work has shown that transformers in simple settings (without language modeling pretraining) implement learning algorithms such as ridge regression and gradient descent (Akyürek et al., 2022; von Oswald et al., 2022; Dai et al., 2022).”
Leveraging large language models for predictive chemistry (2024, positive): “Importantly, we are also not limited to fine-tuning; in Supplementary Note 5, we show that we can even achieve good performance without fine-tuning by incorporating examples directly into the prompt (so-called in-context learning 5,29 , that is, learning during inference time).”
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher (2023, methodology): “While safety alignment for LLMs can help, LLMs remain vulnerable to adversarial inputs that can elicit undesired behavior (Gehman et al., 2020; Bommasani et al., 2021; walkerspider, 2022; Perez et al., 2022; Perez & Ribeiro, 2022; Kang et al., 2023; Li et al., 2023; Ganguli et al., 2022; Schulhoff e”
A Survey of Large Language Models (2023, positive): “In some recent studies [60], it also shows that in-context learning implicitly performs meta-optimization through the attention mechanism.”
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model (2023, methodology): “Due to the pre-training of F LAN -T5 models on a large-scale chain-of-thought-(CoT) and instruction-based dataset, Dai et al. [4] posit that they are able to learn a new task very well from the in-context information by mimicking gradient descent through attention weights.”
Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective (2023, background): “Subsequent works further revealed that Transformers can implement learning algorithms such as linear regression [1], gradient descent [1, 54, 16], and Bayesian inference [59].”
Large language models in medicine (2023, methodology): “The provision of examples by users enables LLMs to train themselves in a process similar to the fine-tuning employed in their initial development 24 .”
“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation (2023, background): “[20] Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei.”
When large language models meet personalization: perspectives of challenges and opportunities (2023, neutral): “In-context learning [148], [149], [150], [151], [152], [153], [154]”
A Survey on In-context Learning (2022, background): “Dai et al. (2022) figured out a dual form between Transformer attention and gradient descent and further proposed to understand ICL as implicit fine-tuning.”
Otter: A Multi-Modal Model With In-Context Instruction Tuning (2023, neutral): “3.3.”
A survey on large language models for recommendation (2023, positive): “Liu et al. (2023a) and Dai et al. (2023) designed the demonstration example templates for various recommendation tasks and the experimental results also showed the in-context learning method will improve the recommendation abilities of LLMs on most tasks.”
Language Models can Solve Computer Tasks (2023, background): “Recently, large language models (LLMs) have shown remarkable in-context learning capabilities across a variety of domains and tasks [10, 53, 4, 14, 20, 49, 7, 35, 5].”