Meta-Learning for Semi-Supervised Few-Shot Classification

By Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Josh Tenenbaum, Hugo Larochelle, Richard S. Zemel. International Conference on Learning Representations, 2018

1,444 citations recorded by the source corpus.

In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for a learning algorithm is defined and trained on episodes representing different classification problems, each with a small labeled training set and its corresponding test set. In this work, we advance this few-shot classification paradigm towards a scenario where unlabeled examples are also available within each episode. We consider two situations: one where all unlabeled examples are assumed to belong to the same set of classes as the labeled examples of the episode, as well as the more challenging situation where examples from other distractor classes are also provided. To address this paradigm, we propose novel extensions of Prototypical Networks (Snell et al., 2017) that are augmented with the ability to use unlabeled examples when producing prototypes. These models are trained in an end-to-end way on episodes, to learn to leverage the unlabeled examples successfully. We evaluate these methods on versions of the Omniglot and miniImageNet benchmarks, adapted to this new framework augmented with unlabeled examples. We also propose a new split of ImageNet, consisting of a large set of classes, with a hierarchical structure. Our experiments confirm that our Prototypical Networks can learn to improve their predictions due to unlabeled examples, much like a semi-supervised algorithm would.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

View Paper: https://arxiv.org/abs/1803.00676

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, methodology): “We conduct experiments on five widely used few-shot image recognition benchmarks: miniImageNet [51], tieredImageNet [40], CIFAR-FS [3], and FC100 [33], and Meta-Dataset [50].”
Meta-learning with differentiable closed-form solvers (2018, background): “Prototypical networks [10, 41] generalize them to k-nearest neighbors.”
Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (2019, neutral): “, 2018), tiered-ImageNet (Ren et al., 2018), and ImageNet-21k (Dhillon et al.”
Meta-Learning with Latent Embedding Optimization (2018, background): “As argued in Ren et al. (2018), this split near the root of the ImageNet hierarchy results in a more challenging, yet realistic regime with test classes that are less similar to training classes.”
Meta-Learning With Differentiable Convex Optimization (2019, methodology): “The tieredImageNet benchmark [23] is a larger subset of ILSVRC-2012 [24], composed of 608 classes grouped into 34 high-level categories.”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, background): “It has also been shown that these approaches benefit from leveraging unlabeled and simulated data [23, 34].”
Towards Label-Efficient Incremental Learning: A Survey (2023, background): “However, a combination of different forms of supervision is common in non-incremental learning, such as the combination of self-and few-shot-supervision [Su et al. , 2020], or semi and few-shot-supervision [Ren et al. , 2018; Li et al. , 2019].”
Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, positive): “It has been shown that existing works [47,35,36] are particularly sensitive to OOD data.”
CoMPS: Continual Meta Policy Search (2021, background): “Recent work has made progress toward using supervised meta-learning in an online setting (Ren et al., 2018; Finn et al., 2019; Khodak et al., 2019; Jerfel et al., 2019; Grant et al., 2019; Zhuang et al., 2019; Antoniou et al., 2020; Wallingford et al., 2020; Yu et al., 2020a; Yao et al., 2020).”
Continual Learning via Local Module Composition (2021, methodology): “The second sequence of environments is constructed from classes of the miniImagenet [79].”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “tiered-ImageNet [38] is a larger subset of Imagenet [8] with categorical splits for for base, validation, and novel classes.”
Incremental Meta-Learning via Indirect Discriminant Alignment (2020, methodology): “For a fair comparison, we use the same training, validation and testing splits of [28] and use the classes at the lowest level of the hierarchy.”
Continual Few-Shot Learning for Text Classification (2021, background): “Few-shot learning problems are studied in the image domain (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017; Ren et al., 2018; Sung et al., 2018), focusing mainly on two kinds of approaches: metric-based approaches and optimization-based approaches.”
Incremental Few-Shot Learning with Attention Attractor Networks (2018, methodology): “…recurrent back-propagation (Liao et al., 2018; Almeida, 1987; Pineda, 1987) we show empirically that our proposed method can produce state-of-the-art results in incremental few-shot learning on the challenging mini -ImageNet (Vinyals et al., 2016) and tiered -ImageNet (Ren et al., 2018) tasks.”
Meta-Learning: A Survey (2018, neutral): “[131] extend this approach to semi-supervised learning.”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, background): “Tiered-ImageNet Like Omniglot, Tiered-ImageNet (Ren et al., 2018) groups classes into super-categories corresponding to higher-level nodes in the ImageNet (Deng et al., 2009) hierarchy (20, 6, and 8 disjoint sets of training, validation, and testing nodes).”
Defining Benchmarks for Continual Few-Shot Learning (2020, methodology): “, 2016) and Tiered-ImageNet (Ren et al., 2018), whereas for the single-incremental-task continual learning setting (Maltoni & Lomonaco, 2019) and the multi-task continual setting (Zenke et al.”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “There are a number of established FSL datasets that are used in this setting, such as miniImageNet [41], tieredIm-ageNet [204], SlimageNet [205], CUB-200 [110] and Om-niglot [86].”