Low-Shot Visual Recognition by Shrinking and Hallucinating Features

By Bharath Hariharan, Ross B. Girshick. 2016

807 citations recorded by the source corpus.

Low-shot visual learning–the ability to recognize novel object categories from very few examples–is a hallmark of human visual intelligence. Existing machine learning approaches fail to generalize in the same way. To make progress on this foundational problem, we present a low-shot learning benchmark on complex images that mimics challenges faced by recognition systems in the wild. We then propose (1) representation regularization techniques, and (2) techniques to hallucinate additional training examples for data-starved classes. Together, our methods improve the effectiveness of convolutional networks in low-shot learning, improving the one-shot accuracy on novel classes by 2.3× on the challenging ImageNet dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

LVIS: A Dataset for Large Vocabulary Instance Segmentation (2019, background): “In the 1k subset, 89% of the categories have more than 20 training instances, while the low-shot literature typically considers 20 examples per category [8].”
A Survey of Data Augmentation Approaches for NLP (2021, background): “ing approaches (Hariharan and Girshick, 2017; Schwartz et al., 2018) leverage estimated feature space "analogy" transformations between examples of known classes to augment for novel classes (see §4.”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “…examples is provided, is referred to as one-shot or few-shot learning (Miller et al., 2000; Fei-Fei et al., 2006; Lake et al., 2015; Hariharan & Girshick, 2017) and has recently seen a tremendous surge in interest within the machine learning community (e.g. Vinyals et al. (2016); Bertinetto et…”
Billion-scale semi-supervised learning for image classification (2019, background): “Typical works on this line of research propose parameter prediction [4, 33] and data augmentation [12, 14].”
Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (2019, background): “The multiplication with β enables the potential generation of smaller support sets even when multiple images are available, since we are also interested in examining the very-low-shot end of the spectrum.”
A Closer Look at Few-shot Classification (2019, methodology): “Examples include model initialization based methods Ravi & Larochelle (2017); Finn et al. (2017), metric learning methods Vinyals et al. (2016); Snell et al. (2017); Sung et al. (2018), and hallucination based methods Antoniou et al. (2018); Hariharan & Girshick (2017); Wang et al. (2018).”
Dynamic Few-Shot Visual Learning Without Forgetting (2018, methodology): “Logistic regression w/ H [4] (from [26]) 40.”
Low-Shot Learning from Imaginary Data (2018, methodology): “We use the benchmark proposed by Hariharan and Girshick [13].”
Few-Shot Image Recognition by Predicting Parameters from Activations (2017, background): “Model Regression [31] and SGM [10] are suitable for both large-scale and few-shot image recognition.”
Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation (2022, background): “It can be roughly divided into metric-based methods [32, 28, 39, 25], meta-learning based methods [12, 23, 27] and augmentation-based methods [36, 14, 1].”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, background): “First work [16, 48] on GFSL propose to hallucinate extra samples based on intra-class variations of base classes.”
Memory-Efficient Incremental Learning Through Feature Adaptation (2020, methodology): “Our feature adaptation method is inspired by the feature hallucinator proposed by Hariharan and Girschick [18].”
Learning to Reconstruct Shapes from Unseen Classes (2018, background): “More recently, many researchers have begun to study zero- or few-shot recognition with deep networks [Akata et al., 2016, Antol et al., 2014, Hariharan and Girshick, 2017, Wang et al., 2017, Wang and Hebert, 2016].”
The limits and potentials of deep learning for robotics (2018, methodology): “This class-incremental learning would preferably be data-efﬁcient by using one-shot [13], [60], [92], [99], [124] or low-shot [27], [41], [126] learning techniques.”
Incremental Few-Shot Learning with Attention Attractor Networks (2018, methodology): “To leverage a good representation, [10, 37, 9] starts off with a pre-trained network on a set of base classes, and tries to augment the classifier with a batch of new classes that has not been seen during training.”
Learning to Generalize: Meta-Learning for Domain Generalization (2017, neutral): “Related Methods Related to MLDG-GN, squared gradient magnitude loss (SGM) was concurrently proposed in (Hariharan and Girshick 2017) for few-shot recognition.”
Large-Scale Long-Tailed Recognition in an Open World (2019, methodology): “Traditional deep learning models are good at capturing the big data of head classes [24, 18]; more recently, few-shot learning methods have been developed for the small data of tail classes [48, 16].”