SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning

By Yan Wang, Wei-Lun Chao, Kilian Q. Weinberger, L. V. D. van der Maaten. 2019

388 citations recorded by the source corpus.

Few-shot learners aim to recognize new object classes based on a small number of labeled training examples. To prevent overfitting, state-of-the-art few-shot learners use meta-learning on convolutional-network features and perform classification using a nearest-neighbor classifier. This paper studies the accuracy of nearest-neighbor baselines without meta-learning. Surprisingly, we find simple feature transformations suffice to obtain competitive few-shot learning accuracies. For example, we find that a nearest-neighbor classifier used in combination with mean-subtraction and L2-normalization outperforms prior results in three out of five settings on the miniImageNet dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “, prototypical network [62], [87] shows competitive performance in resist overfitting.”
Defining Benchmarks for Continual Few-Shot Learning (2020, methodology): “Similarly, a method based on nearest neighbor classifier has recently showed to achieve state-of-the-art performances (Wang et al., 2019).”
Meta-Learning in Neural Networks: A Survey (2020, background): “On a similar theme, [214] show that simple feature transformations like L2-normalization can make a nearest neighbour classiﬁer competitive without meta-learning.”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, neutral): “…ResNet-12 65.60±1.80 78.60±0.90 MetaOptNet (Lee et al. 2019) ResNet-12 65.99±0.72 81.56±0.63 Feat (Ye et al. 2020) ResNet-12 70.80±0.23 84.79±0.16 SimpleShot (Wang et al. 2019) ResNet-18 69.09±0.22 84.58±0.16 CTM (Li et al. 2019) ResNet-18 68.41±0.39 84.28±1.73 LEO (Rusu et al. 2019) WRN…”
Interventional Few-Shot Learning (2020, methodology): “Our implementation of k-NN is similar to [57, 68].”
Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions (2018, methodology): “Additionally, we list some recent methods (SimpleShot [45], and CTM [23]) using different backbone architectures such as ResNet-18 for reference.”
Heterogeneous Few-Shot Model Rectification With Semantic Mapping (2020, neutral): “observation in a concurrent paper [74].”
Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels (2020, background): “Where BNCosSim is a variant of CosSim with features centered through BatchNorm (BN) statistics (Ioffe and Szegedy, 2015), this has shown to improve performance (Wang et al., 2019).”
How to Train Your MAML to Excel in Few-Shot Classification (2021, background): “dedicated to, MAML’s performance has been shown to fall far behind [14, 94, 104], even after paired with a stronger backbone, e.”
Generalized Many-Way Few-Shot Video Classification (2020, background): “Another approach is to cast low-shot learning as a nearest-neighbor classiﬁer [44].”
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning (2021, neutral): “63 SimpleShot [50] ResNet-18 Metric 62.”
Leveraging the Feature Distribution in Transfer-based Few-Shot Learning (2020, methodology): “Multiple works in the domain [34, 14] discuss the different statistical methods (e.”
Are We Overfitting to Experimental Setups in Recognition (2020, background): “*Equal contribution Recent works have raised concerns about the progress in few-shot and continual learning by presenting simple baselines that are comparable to state of the art in each field [32, 6, 48, 44].”
Few-Shot Learning With a Strong Teacher (2021, methodology): “Even in the standard 5-way 5-shot FSL setting, meta-learning methods [14] can be outperformed by a simple nearest centroid classifier [9], [18] whose features are trained plainly with the base class data; the gap becomes more pronounced in a larger shot setting where the query set size shrinks3.”
Baby steps towards few-shot learning with multiple semantics (2019, methodology): “3 – – SimpleShot [33] 64.”