A Baseline for Few-Shot Image Classification

By Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, Stefano Soatto. 2019

646 citations recorded by the source corpus.

Fine-tuning a deep network trained with the standard cross-entropy loss is a strong baseline for few-shot learning. When fine-tuned transductively, this outperforms the current state-of-the-art on standard datasets such as Mini-ImageNet, Tiered-ImageNet, CIFAR-FS and FC-100 with the same hyper-parameters. The simplicity of this approach enables us to demonstrate the first few-shot learning results on the ImageNet-21k dataset. We find that using a large number of meta-training classes results in high few-shot accuracies even for a large number of few-shot classes. We do not advocate our approach as the solution for few-shot learning, but simply use the results to highlight limitations of current benchmarks and few-shot protocols. We perform extensive studies on benchmark datasets to propose a metric that quantifies the "hardness" of a few-shot episode. This metric can be used to report the performance of few-shot algorithms in a more systematic way.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, methodology): “Recently, [9] proposed transductive fine-tuning as a strong baseline for few-shot classification; and even in a regular, inductive, few-shot setup, they showed that fine-tuning is only slightly worse than state-of-the-art algorithms.”
CrossTransformers: spatially-aware few-shot transfer (2020, neutral): “[20] - - 68.”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “In FSL [9, 34] and IL [26, 36] multiple works show that adaptive representations can be beneficial for learning novel classes, specifically to fine-tune the parameters of the representation.”
Incremental Meta-Learning via Indirect Discriminant Alignment (2020, background): “This is not the case for few-shot learning, where the current state-of-the-art still lags far behind [5].”
ImageNet-21K Pretraining for the Masses (2021, background): “There are also works on ImageNet-21K that did not focus on pretraining: [53] used extra (unlabled) data from ImageNet-21K to improve knowledge-distillation training on ImageNet-1K; [11] used ImageNet-21k for testing few-shot learning; [48] tested efficient softmax schemes on ImageNet-21k; [15] teste”
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling (2021, methodology): “Constructing a cache model from few-shot training data has been explored by many previous methods, including Matching Network [63], Prototypical Networks [55], MAML [16], Relation Network [58] and others [6,7,12,60].”
Interventional Few-Shot Learning (2020, methodology): “1, 5 [18] Guneet S Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto.”
LEEP: A New Measure to Evaluate Transferability of Learned Representations (2020, methodology): “Another popular transfer learning method, called the ﬁne-tuning method, ﬁne-tunes the feature extractor while re-training the new head classiﬁer on the target data set to get a model for the target task (Agrawal et al., 2014; Girshick et al., 2014; Chatﬁeld et al., 2014; Dhillon et al., 2020).”
A general approach to progressive learning (2020, background): “This extends previously proposed approaches which only transferred forwards, from past to future tasks [10, 29].”
Scaling Vision with Sparse Mixture of Experts (2021, methodology): “We consider two different approaches to Transfer Learning: linear few-shot learning on fixed representations and full fine-tuning of the model.”
FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, result): “8 These results are similar to recent trends in the field of meta-learning, where fine-tuning based on well-trained representations overwhelms advanced few-shot classification algorithms for heterogeneous tasks (Chen et al., 2019a; 2020a; Dhillon et al., 2019; Tian et al., 2020).”
On Episodes, Prototypical Networks, and Few-shot Learning (2020, methodology): “, 2019), transductive ne-tuning (Dhillon et al., 2020) or logistic regression (Tian et al.”
Meta-Learning with Shared Amortized Variational Inference (2020, neutral): “SAMOVAR yields the best results on the 5-shot (Dhillon et al., 2020) reports a higher accuracy for the 1-shot setting, but is not directly comparable due to the transductive nature of their approach.”