Low-Shot Learning from Imaginary Data

By Yu-Xiong Wang, Ross B. Girshick, Martial Hebert, Bharath Hariharan. 2018

735 citations recorded by the source corpus.

Humans can quickly learn new visual concepts, perhaps because they can easily visualize or imagine what novel objects look like from different views. Incorporating this ability to hallucinate novel instances of new concepts might help machine vision systems perform better low-shot learning, i.e., learning concepts from few examples. We present a novel approach to low-shot learning that uses this idea. Our approach builds on recent progress in meta-learning ("learning to learn") by combining a meta-learner with a "hallucinator" that produces additional training examples, and optimizing both models jointly. Our hallucinator can be incorporated into a variety of meta-learners and provides significant gains: up to a 6 point boost in classification accuracy when only a single training example is available, yielding state-of-the-art performance on the challenging ImageNet low-shot classification benchmark.

Subfields: Artificial Intelligence, Media Technology, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Media Technology

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, background): “Recently, significant advances [56, 55, 53, 12, 46, 48, 57, 34, 44, 62, 26, 28] have been made to tackle this problem using the ideas of meta-learning or “learning to learn”.”
Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (2019, background): “, 2016), some learn to perform data augmentation (Hariharan & Girshick, 2017; Wang et al., 2018) in a low-shot learning setting, and some learn to predict the parameters of a large-shot classifier from the parameters learned in a few-shot setting (Wang & Hebert, 2016; Wang et al.”
A Closer Look at Few-shot Classification (2019, methodology): “Examples include model initialization based methods Ravi & Larochelle (2017); Finn et al. (2017), metric learning methods Vinyals et al. (2016); Snell et al. (2017); Sung et al. (2018), and hallucination based methods Antoniou et al. (2018); Hariharan & Girshick (2017); Wang et al. (2018).”
Dynamic Few-Shot Visual Learning Without Forgetting (2018, background): “We repeat the above experiment 100 times (sampling each time a different set of training images for the novel categories) and report in Table 3 the mean accuracies and the 95% confidence intervals for the recognition accuracy metrics proposed in [26].”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, background): “It has also been shown that these approaches benefit from leveraging unlabeled and simulated data [23, 34].”
Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation (2022, background): “It can be roughly divided into metric-based methods [32, 28, 39, 25], meta-learning based methods [12, 23, 27] and augmentation-based methods [36, 14, 1].”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “The first phase is devoted to general representation learning as in previous work [14, 37, 48].”
Incremental Few-Shot Learning with Attention Attractor Networks (2018, methodology): “To leverage a good representation, [10, 37, 9] starts off with a pre-trained network on a set of base classes, and tries to augment the classifier with a batch of new classes that has not been seen during training.”
Meta-Transfer Learning for Few-Shot Learning (2018, background): “alternative is to merge data from multiple tasks which, however, is often not effective due to variances of the data across tasks [46].”
Large-Scale Long-Tailed Recognition in an Open World (2019, methodology): “Following [16, 53, 13], we employ the scratch ResNet-10 [18] as our backbone network for ImageNet-LT.”
Decoupling Representation and Classifier for Long-Tailed Recognition (2019, background): “Another direction is to enhance recognition performance of the tail classes by transferring knowledge from the head classes (Wang et al., 2017; 2018; Zhong et al., 2019; Liu et al., 2019).”
Few-Shot Object Detection via Feature Reweighting (2018, background): “[16, 44] introduces image hallucination techniques to augment the training data and improve generalization from base categories to few-shot novel categories.”
Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning (2019, methodology): “Some of the previous GFSL approaches (Hariharan and Girshick 2017; Wang et al. 2018; Gao et al. 2018) apply the exemplar-based classification paradigms on both seen and unseen categories to resolve the transductive learning problem, which requires recomputing the centroids for seen categories after…”
Learning to Self-Train for Semi-Supervised Few-Shot Classification (2019, neutral): “They can be roughly divided into four categories: (1) data augmentation based methods [15, 29, 37, 38] generate data or features in a conditional way for few-shot classes; (2) metric learning methods [36, 31, 33] learn a similarity space of image features in which the classification should be effici”
Meta-Curvature (2019, background): “In order to overcome the nature of few-shot learning, the generative models have been suggested to augment the training data [40, 45] or generate model parameters for the speciﬁed task [39, 33].”
Boosting Few-Shot Visual Learning With Self-Supervision (2019, neutral): “, training images, of a few-shot task [26, 33, 53, 55, 57, 59]; methods learning to map a test example to a class label by accessing memory modules that store training examples for that task [11, 25, 35, 38, 50]; approaches that learn how to generate the weights of a classifier [12, 16, 43, 44, 13] ”
Generalizing from a Few Examples (2019, methodology): “PMN [141] CNN, LSTM CNN, biLSTM cosine similarity”
Meta-Transfer Learning Through Hard Tasks (2019, background): “ditioned on Gaussian noises [11]–[13] or object attributes [14].”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, background): “Research literature on few-shot learning paradigms exhibits a high diversity from using data augmentation techniques [9, 75,77] over sharing feature representation [2,76] to meta-learning [18,72].”