Delta-encoder: an effective sample synthesis method for few-shot object recognition

By Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogério Schmidt Feris, Raja Giryes, A. Bronstein. 2018

387 citations recorded by the source corpus.

Learning to classify new categories based on just one or a few examples is a long-standing challenge in modern computer vision. In this work, we proposes a simple yet effective method for few-shot (and one-shot) object recognition. Our approach is based on a modified auto-encoder, denoted Delta-encoder, that learns to synthesize new samples for an unseen category just by seeing few examples from it. The synthesized samples are then used to train a classifier. The proposed approach learns to both extract transferable intra-class deformations, or "deltas", between same-class pairs of training examples, and to apply those deltas to the few provided examples of a novel class (unseen during training) in order to efficiently synthesize samples from that new class. The proposed method improves over the state-of-the-art in one-shot object-recognition and compares favorably in the few-shot case. Upon acceptance code will be made available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

A Survey of Data Augmentation Approaches for NLP (2021, background): “Many few-shot learning approaches (Hariharan and Girshick, 2017; Schwartz et al., 2018) leverage estimated feature space "analogy" transformations between examples of known classes to augment for novel classes (see §4.4).”
Meta-Transfer Learning for Few-Shot Learning (2018, background): “nseethat aconsistent gain of 1% is achievedby using HT meta-batch. 7 Few-shot learning method Feature extractor 1-shot 5-shot Data augmentation Adv. ResNet, [25] WRN-40 (pre) 55.2 69.6 Delta-encoder, [38] VGG-16 (pre) 58.7 73.6 Metric learning Matching Nets, [45] 4 CONV 43.44 ± 0.77 55.31 ± 0.73 Pro”
Learning to Self-Train for Semi-Supervised Few-Shot Classification (2019, background): “They are based on either data augmentation [15, 29] or gradient descent [3, 23, 7, 5, 42, 19, 27, 32, 10].”
Meta-Curvature (2019, methodology): “In order to overcome the nature of few-shot learning, the generative models have been suggested to augment the training data (Schwartz et al., 2018; Wang et al., 2018) or generate model parameters for the specified task (Rusu et al.”
Generalizing from a Few Examples (2019, background): “Similarly, a set of auto-encoders, each representing one intra-class variability, are learned from similar classes in Reference [116].”
Meta-Transfer Learning Through Hard Tasks (2019, neutral): “6 Delta-encoder, [12] VGG-16 (pre) 58.”
RepMet: Representative-Based Metric Learning for Classification and Few-Shot Object Detection (2018, positive): “Recently, even more compelling results were obtained on the standard few-shot classification benchmarks using meta-learning methods [9, 19, 24, 43] and synthesis methods [6, 10, 29, 40, 43].”
Charting the Right Manifold: Manifold Mixup for Few-shot Learning (2019, background): “Hallucination based methods [20, 65, 56] augment the limited training data for a new task by generating or hallucinating new data points.”
Free Lunch for Few-shot Learning: Distribution Calibration (2021, background): “, 2019b) or encoding the intra-class deformations (Schwartz et al., 2018).”
DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning (2020, background): “classification problem, such as works based on graph theories [66], [67], [68], [69], reinforcement learning [70], differentiable SVM [71], generative models [72], [73], [74], [75], [76], [77], [78], [79], [80], transductive learning [81], [82], [83], [84], [85], recurrent models [86], [87], self-su”
A Broader Study of Cross-Domain Few-Shot Learning (2019, background): “In [19,52] the intra-class relations between pairs of instances of reference categories are modeled in feature space, and then this information is transferred to the novel category instances to generate additional examples in that same feature space.”
One-Shot Instance Segmentation (2018, methodology): “Other approaches are based on generative models [38, 76], ideas from information retrieval [81] or employ meta learning [24, 40, 52, 51, 53, 54, 74, 80, 69].”
Large-Scale Few-Shot Learning: Knowledge Transfer With Class Hierarchy (2019, methodology): “More recently, methods based on feature hallucination and synthesis [7, 23] or predicting parameters of the network [20, 19] have been developed.”
Image Deformation Meta-Networks for One-Shot Learning (2019, methodology): “As summarized in Table 4, we mainly focus on three groups of competitors: (1) meta-learning algorithms, such as MAML [5] and Meta-SGD [13]; (2) metric learning algorithms, including matching networks [28], prototypical networks [22], relation networks [23], SNAIL [14], deltaencoder [21], and Cosine ”
Feature Weighting and Boosting for Few-Shot Segmentation (2019, background): “Prior work can be broadly divided into three groups: transfer learning of models trained on classes similar to the target classes [28, 22, 29, 30], meta-learning approaches that learn how to effectively learn new classes from small datasets [8, 18], and generative approaches aimed at data-augmentati”
Baby steps towards few-shot learning with multiple semantics (2019, methodology): “7 ∆ -encoder [44] 59 .”
Revisiting Fine-tuning for Few-shot Learning (2019, methodology): “Some methods synthesize examples of novel classes by applying within-class differences of base classes to real examples of novel classes (Hariharan & Girshick, 2017; Schwartz et al., 2018).”