DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning

By Chi Zhang, Yujun Cai, Guosheng Lin, Chunhua Shen. 2020

186 citations recorded by the source corpus.

In this work, we develop methods for few-shot image classification from a new perspective of optimal matching between image regions. We employ the Earth Mover's Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to calculate the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively alleviate the adverse impact caused by the cluttered background and large intra-class appearance variations. To implement <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="lin-ieq1-3217373.gif"/></alternatives></inline-formula>-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit function theorem, the EMD can be inserted as a layer into the network for end-to-end training. Our extensive experiments validate the effectiveness of our algorithm which outperforms state-of-the-art methods by a significant margin on five widely used few-shot classification benchmarks, namely, miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100), Caltech-UCSD Birds-200-2011 (CUB), and CIFAR-FewShot (CIFAR-FS). We also demonstrate the effectiveness of our method on the image retrieval task in our experiments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chi Zhang
2. Guosheng Lin
3. Qixiang Ye
4. Jianbin Jiao
5. Xiangyang Ji
6. Yao Ding
7. Binghao Liu
8. Pan Pan

In Computer Vision and Pattern Recognition

1. Qixiang Ye
2. Guosheng Lin
3. Chi Zhang
4. Jianbin Jiao
5. Xiangyang Ji
6. Bingchen Zhao
7. Pan Pan
8. Yao Ding

Citations in the corpus

• Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima (2021, methodology): “Current few-shot methods mainly include optimization-based methods [12, 23, 32, 39, 45, 46, 55] and metric-based methods [13, 19, 44, 49, 52, 58, 57, 53].”
• Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “We first consider four kinds of classifiers, including the vanilla linear classifier in the CNNs, the cosine classifier [42], the L2 classifier [35], and the DeepEMD classifier [52], where their main difference is the metric to compute class scores given the prototypes of each class.”
• Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, background): “While meta-learning [36, 27, 8, 15, 44, 38, 21] contributed important optimization methods and data augmentation [13, 35] aggregated performance, metric learning [32, 30, 11, 5] with prototype models [23, 4, 6, 41, 40, 18] represent the majority of few-shot learning approaches.”
• Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, background): “Verification based on part-level features has been shown to be effective to match two images ( e.g. [55, 12]).”