Revisiting Local Descriptor Based Image-To-Class Measure for Few-Shot Learning

By Wenbin Li, Jinglin Xu, Jing Huo, Yang Gao, Jiebo Luo. Computer Vision and Pattern Recognition, 2019

591 citations recorded by the source corpus.

Few-shot learning in image classification aims to learn a classifier to classify images when only few training examples are available for each class. Recent work has achieved promising classification performance, where an image-level feature based measure is usually used. In this paper, we argue that a measure at such a level may not be effective enough in light of the scarcity of examples in few-shot learning. Instead, we think a local descriptor based image-to-class measure should be taken, inspired by its surprising success in the heydays of local invariant features. Specifically, building upon the recent episodic training mechanism, we propose a Deep Nearest Neighbor Neural Network (DN4 in short) and train it in an end-to-end manner. Its key difference from the literature is the replacement of the image-level feature based measure in the final layer by a local descriptor based image-to-class measure. This measure is conducted online via a k-nearest neighbor search over the deep local descriptors of convolutional feature maps. The proposed DN4 not only learns the optimal deep local descriptors for the image-to-class measure, but also utilizes the higher efficiency of such a measure in the case of example scarcity, thanks to the exchangeability of visual patterns across the images in the same class. Our work leads to a simple, effective, and computationally efficient framework for few-shot learning. Experimental study on benchmark datasets consistently shows its superiority over the related state-of-the-art, with the largest absolute improvement of 17% over the next best. The source code can be available from https://github.com/WenbinLee/DN4.git.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/821c0d99eec8e2ee312e08b5605924e284b7cc7c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

MeLL: Large-scale Extensible User Intent Classification for Dialogue Systems with Meta Lifelong Learning (2021, background): “Typical applications include few-shot image classification [25], objection detection [12] and many others.”
Hyperbolic Image Embeddings (2019, background): “Most of the existing state-ofthe-art few–shot learning models are based on metric learning approaches, utilizing the distance between image representations computed by deep neural networks as a measure of similarity [53, 43, 48, 28, 4, 6, 23, 2, 38, 5].”
DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers (2020, methodology): “Our method is more related to the metric-based methods [30, 43, 63, 66, 67, 73], which aim to represent samples in an appropriate feature space where data from different categories can be distinguished with distance metrics.”
Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector (2019, background): “A number of works [41, 42] exploit local descriptors to reap additional knowledge from limited data.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, unknown): “38% ✓ DN4 [243] CVPR’19 Meta Conv-64 ✓ ✓ ✓ 51.”
Self-Supervised Learning for Few-Shot Image Classification (2019, unknown): “…[5] 4 Conv 49.42 ± 0.78% 68.20 ± 0.66% Baseline* [28] 4 Conv 41.08 ± 0.70% 54.50% ± 0.66 Spot&learn [29] 4 Conv 51.03 ± 0.78% 67.96% ± 0.71 DN4 [30] 4 Conv 51.24 ± 0.74% 71.02% ± 0.64 SNAIL [31] ResNet12 55.71 ± 0.99% 68.88 ± 0.92% ProtoNet + [5] ResNet12 56.50 ± 0.40% 74.2 ± 0.20% CAML [9]…”
Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search (2020, methodology): “image during inference. 9 Table V COMPARISON OF TIGDA WITH ONE-SHOT LEARNING METHODS. Method A!B C!B ProtoNet [52] 61.9 0.1 49.6 0.3 MatchingNet [52] 57.6 0.2 43.7 0.1 DAPN [53] 68.9 0.2 51.9 0.2 DN4 [54] 55.4 0.1 44.6 0.2 FADA [55] 60.6 0.3 45.9 0.3 TIGDA (Ours) 76.2 0.3 56.2 0.3 C. Ablation studie”
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning (2021, methodology): “A few methods have also been proposed that aim to identify finer-grained local features, such as DN4 [24], SAML [10], STANet [54] and CTM [23].”
Adversarial Feature Hallucination Networks for Few-Shot Learning (2020, unknown): “v-64F NeurIPS’17 49.42 0.78 68.20 0.66 MM-Net [2] Conv-64F CVPR’18 53.37 0.48 66.97 0.35 GNN [11] Conv-256F Arxiv’17 50.33 0.36 66.41 0.63 RELATION NET [37] Conv-64F CVPR’18 50.44 0.82 65.32 0.70 DN4 [23] Conv-64F CVPR’19 51.24 0.74 71.02 0.64 TPN [25] ResNet8 ICLR’19 55.51 0.86 69.86 0.65 PARN [43]”
Generalized Adaptation for Few-Shot Learning (2019, unknown): “˚ (˝ sj˚)(˚ 0 ˚) + L(˝j˚)] (15) We take Eq. 12 into Eq. 15 then the objective of Reptile is: min ˚ E[ L0 ˚ (˝ sj˚)L 0 ˚ (˝j˚) + L(˝j˚)] (16) The rst term can be further simplied as: kL0 ˚ (˝ sj˚)k22 (17) The overall objective of Reptile can be formulated as: min ˚ E[ kL0 ˚ (˝ sj˚)k 2 2 + L(˝j˚)] (18”
DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning (2020, background): “[6] adopt k-NN to fuse local distances.”
Hierarchical Graph Attention Network for Few-shot Visual-Semantic Learning (2021, methodology): “Baselines FPAIT [11] directly leverages MAML [13] to deal with few-shot VQA and IC tasks; Prototypical Net [42], Relation Net [44], R2D2 [6], and DN4 [28] focus on few-shot classi ﬁ cation.”
Cross Attention Network for Few-shot Classification (2019, methodology): “We use the standard split following [28, 34, 24, 13, 30]: 64 classes for training, 16 for validation and 20 for testing.”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, background): “To resolve the generalization problem, many recent approaches to image classification made various attempts in training deep networks with a few annotated examples [1, 18, 25, 31, 50, 53, 59, 65, 67, 73, 79, 84, 85].”
Curriculum-Based Meta-learning (2021, methodology): “Meta-learning methods can be cast into two main categories: model-initialization based methods [3, 10, 33], andmetric-learning basedmethods [14, 24, 36, 39, 41].”