Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

By Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang. 2022

972 citations recorded by the source corpus.

When transferring a pretrained model to a downstream task, two popular methods are full fine-tuning (updating all the model parameters) and linear probing (updating only the last linear layer—the “head”). It is well known that fine-tuning leads to better accuracy in-distribution (ID). However, in this paper, we find that fine-tuning can achieve worse accuracy than linear probing out-of-distribution (OOD) when the pretrained features are good and the distribution shift is large. On 10 distribution shift datasets (Breeds-Living17, Breeds-Entity30, DomainNet, CIFAR → STL, CIFAR10.1, FMoW, ImageNetV2, ImageNet-R, ImageNet-A, ImageNet-Sketch), fine-tuning obtains on average 2% higher accuracy ID but 7% lower accuracy OOD than linear probing. We show theoretically that this tradeoff between ID and OOD accuracy arises even in a simple setting: fine-tuning overparameterized two-layer linear networks. We prove that the OOD error of fine-tuning is high when we initialize with a fixed or random head—this is because while fine-tuning learns the head, the lower layers of the neural network change simultaneously and distort the pretrained features. Our analysis suggests that the easy two-step strategy of linear probing then full fine-tuning (LP-FT), sometimes used as a fine-tuning heuristic, combines the benefits of both fine-tuning and linear probing. Empirically, LP-FT outperforms both fine-tuning and linear probing on the above datasets (1% better ID, 10% better OOD than full fine-tuning).

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “The ﬁrst method initializes the model from a linear probe (LP), as described in Kumar et al. (2022), and we refer to this method as LP initialization.”
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? (2023, background): “, 2022), to selectively updating some or all weights of the backbone (Kumar et al., 2022; Hansen et al., 2022a; Yadav et al., 2023).”
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (2023, methodology): “Finetuning typically improves in-distribution generalization, but often performs poorly out-of-distribution, sometimes even degrading performance relative to zero-shot prompting (Kumar et al., 2022; Wortsman et al., 2022b; Awadalla et al., 2022).”
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift (2022, background): “Kumar et al. [2022] use the lightweight ﬁne-tuning intervention.”
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations (2022, background): “28, 41, 34, 93, 61, 49, 60], and study when and why transfer learning can be effective [104, 69, 1, 51, 54].”
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads (2024, background): “Therefore, higher temperatures should result in more opportunities for the original model to accept the draft model’s output.”
A Unified Continual Learning Framework with General Parameter-Efficient Tuning (2023, positive): “Kumar et al. [22] showcase how the linear probing followed by fine-tuning strategy effectively balances performance across out-of-distribution and in-distribution tasks.”
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models (2022, background): “This observation is in alignment with previous work that suggests fine-tuning the image encoder can distort pre-trained features [73, 56].”
Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning (2023, background): “However, conventional fine-tuning might not consistently enhance performance on downstream tasks [25].”
How Well Do Sparse ImageNet Models Transfer? (2021, background): “(A very recent study [39] suggests that this may be inverted on out-of-distribution tasks.”
Fine-Tuning Language Models with Just Forward Passes (2023, background): “We see that (1) ZO-Adam sometimes outperforms ZO-SGD but is not consistent across tasks; (2) LP and then MeZO, as suggested for fine-tuning [51], can sometimes improve the performance.”
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022, background): “However, these approaches are still prone to forgetting prior knowledge, or face difficulties in accessing it concurrently with newly learned concepts (Kumar et al., 2022; Cohen et al., 2022).”
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, background): “Kumar et al. (2022) have showed that fine-tuning does not work well for OOD detection if this data distribution shift is large.”
Transfer Visual Prompt Generator across LLMs (2023, background): “The possible reason can be that, the VPG is typically a pre-trained model with powerful visual perception ability, and thus updating based on the gradient passed through a random projector will mislead the VPG at the initial steps [2, 33, 23].”
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning (2021, neutral): “[17] Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang.”
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2022, background): “It was proven that a two-step strategy with linear probing followed by fine-tuning can outperform only doing fine-tuning directly (Kumar et al., 2022).”