Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

By Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel. 2022

1,382 citations recorded by the source corpus.

Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input. ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.) offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. Along the way, we introduce a new PEFT method called (IA)$^3$ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. We also propose a simple recipe based on the T0 model called T-Few that can be applied to new tasks without task-specific tuning or modifications. We validate the effectiveness of T-Few on completely unseen tasks by applying it to the RAFT benchmark, attaining super-human performance for the first time and outperforming the state-of-the-art by 6% absolute. All of the code used in our experiments is publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

In Optimization & Training

Citations in the corpus

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023, methodology): “The resulting Flan-T5 model converges faster and at a higher performance than T5 models in single-task finetuning—suggesting instruction-tuned models offer a more computationally-efficient starting checkpoint for downstream applications, corroborating Aribandi et al. (2021); Liu et al. (2022b).”
QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “…(PEFT) methods have been proposed such as prompt tuning [49, 33, 34], tuning the embedding layer inputs [1], tuning hidden states (IA 3 ) [37], adding full layers [27], tuning biases [71], learning a mask over weights based on Fisher information [55], and a combination of approaches [23].”
TIES-Merging: Resolving Interference When Merging Models (2023, methodology): “Specifically, we focus on (IA) 3 [43], a PEFT method that scales the base model activations with learned vectors.”
In-context Examples Selection for Machine Translation (2022, methodology): “We propose a novel recall-based re-ranking approach to utilize similar training examples as prompts and show their efﬁcacy across multiple datasets and domains.”
Towards Agile Text Classifiers for Everyone (2023, background): “Recent work suggests that PET may also enable more data-efficient training of models (Liu et al., 2022; Agrawal et al., 2022).”
Generate rather than Retrieve: Large Language Models are Strong Context Generators (2022, background): “A line of recent work has shown that relevant knowledge can be elicited from large language models, especially for those domains that lack appropriate knowledge bases with sufficient coverage (Liu et al., 2022; Fang et al., 2022).”
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, neutral): “Pretrained LLMs are few-shot learners (Brown et al., 2020; Liu et al., 2022; Mosbach et al., 2023).”
Prompt Injection: Parameterization of Fixed Inputs (2022, neutral): “(1) During inference, prompts are always included in the input text and thus incur computational and memory overhead [16].”
A Survey on Model Compression for Large Language Models (2023, positive): “On the other hands, PEQA [Kim et al. , 2023a] and QLORA [Dettmers et al. , 2023a] both fall under the category of quantization-aware Parameter-Efficient Fine-Tuning (PEFT) techniques [Liu et al. , 2022; Ding et al. , 2023; Fu et al. , 2023b].”
Automatic Chain of Thought Prompting in Large Language Models (2022, background): “Despite the success of ICL, studies [Liu et al., 2022a, Lu et al., 2022] have shown that the strength of ICL may vary widely depending on the choice of in-context demonstrations [Liu et al., 2022b].”
LoRA+: Efficient Low Rank Adaptation of Large Models (2024, background): “…prompt tuning (Lester et al., 2021) where a “soft prompt" is learned and appended to the input, the adapters method (Houlsby et al., 2019) where lightweight “adapter" layers are inserted and trained, and ( IA ) 3 (Liu et al., 2022) where activation vectors are modified with learned scalings.”
CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation (2024, positive): “Although zero-shot and few-shot prompting methods provide some degree of adaptability (Dong et al., 2024), task-specific fine-tuning generally delivers better performance, particularly for specialized and out-of-domain tasks (Liu et al., 2022).”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, background): “For example, (IA) 3 [116] introduces three learnable rescaling vectors: l k P R d k , l v P R d v , and l ff P R d ff , to rescale the key, value, and FFN activations, respectively, as depicted in Figure 6 (a).”
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment (2023, methodology): “2) PEFT Methods: Eleven representative PEFT methods: sequential adapter (Adapter S ) [9], prompt-tuning [24], prefix-tuning [10], (IA) 3 [30], BitFit [34], Child-Tuning [39], LoRA [11], AdaLoRA [45], QLoRA [49], MAM adapter [16], and ProPELT [65] are chosen.”