In-Context Learning Creates Task Vectors
By Roee Hendel, Mor Geva, Amir Globerson. Conference on Empirical Methods in Natural Language Processing, 2023
320 citations recorded by the source corpus.
In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In particular, it is challenging to map it to the"standard"machine learning framework, where one uses a training set $S$ to find a best-fitting function $f(x)$ in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query $x$ and a single"task vector"calculated from the training set. Thus, ICL can be seen as compressing $S$ into a single task vector $\boldsymbol{\theta}(S)$ and then using this task vector to modulate the transformer to produce the output. We support the above claim via comprehensive experiments across a range of models and tasks.
Subfields: Artificial Intelligence, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/297211bc86653d9ebbe694a75141c9a1c6c11e69
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Efstratios Gavves
- Leonard Bereska
- David Kristjanson Duvenaud
- Rylan Schaeffer
- Carson E. Denison
- Jared Kaplan
- Karina Nguyen
- Meg Tong
In Management Science and Operations Research
Citations in the corpus
- The Linear Representation Hypothesis and the Geometry of Large Language Models (2023, background): “This is particularly prominent in mechanistic interpretability (Elhage et al., 2021; Meng et al., 2022; Hernandez et al., 2023; Turner et al., 2023; Zou et al., 2023; Todd et al., 2023; Hendel et al., 2023).”
- Many-shot Jailbreaking (2024, background): “This is a simplification of the ”task vector” or ”function vector” findings of Hendel et al. (2023) and Todd et al. (2023).”
- Mechanistic Interpretability for AI Safety - A Review (2024, background): “Causal interventions can extract a high-level understanding of computations and representations learned by large language models (Variengien & Winsor, 2023; Hendel et al., 2023; Feng & Steinhardt, 2023; Zou et al., 2023).”