DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers
By Chi Zhang, Yujun Cai, Guosheng Lin, Chunhua Shen. 2020
903 citations recorded by the source corpus.
In this paper, we address the few-shot classification task from a new perspective of optimal matching between image regions. We adopt the Earth Mover's Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to represent the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively minimize the impact caused by the cluttered background and large intra-class appearance variations. To handle k-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit function theorem, the EMD can be inserted as a layer into the network for end-to-end training. We conduct comprehensive experiments to validate our algorithm and we set new state-of-the-art performance on four popular few-shot classification benchmarks, namely miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100) and Caltech-UCSD Birds-200-2011 (CUB).
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Bernt Schiele
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Chunhua Shen
- Yi Yang
- Ming-Hsuan Yang
In Meta & Continual Learning
Citations in the corpus
- Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation (2022, methodology): “It can be roughly divided into metric-based methods [32, 28, 39, 25], meta-learning based methods [12, 23, 27] and augmentation-based methods [36, 14, 1].”
- Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, methodology): “Besides, we also compare to current state-of-the-art FSCIL algorithms: TOPIC [23], Decoupled-DeepEMD/Cosine [60], [63] and CEC [24].”
- Constrained Few-shot Class-incremental Learning (2022, methodology): “For the natural image datasets, we use a Resnet-12 architecture as feature extractor [11,55].”
- Forward Compatible Few-Shot Class-Incremental Learning (2022, methodology): “• Decoupled-DeepEMD [11]: decouples the training process of embedding and classifier.”
- Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima (2021, methodology): “Current few-shot methods mainly include optimization-based methods [12, 23, 32, 39, 45, 46, 55] and metric-based methods [13, 19, 44, 49, 52, 58, 57, 53].”
- Learngene: From Open-World to Your Learning Task (2021, methodology): “In order to ensure a fair comparison, we reset the capacity of the architectures of eight representative methods ((Finn, Abbeel, and Levine 2017; Nichol, Achiam, and Schulman 2018; Snell, Swersky, and Zemel 2017; Triantafillou et al. 2020; Vinyals et al. 2016; Zhang et al. 2020), two fine-tuning met”
- Adaptive Aggregation Networks for Class-Incremental Learning (2020, background): “Technically, the network parameters are updated at one level and the key hyperparameters are updated at another level [13,21,24,45,46,52].”
- Mnemonics Training: Multi-Class Incremental Learning Without Forgetting (2020, background): “Meta-learning [5, 15, 32, 33, 37, 38] is another BOP in which a meta-learner is optimized subject to the optimality of the base-learner.”
- An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, background): “(1) Metric learning methods [12, 24, 40, 41, 64, 71, 73, 78, 82] aim to learn a similarity space, in which the learning should be efficient for few-shot examples.”
- Interventional Few-Shot Learning (2020, methodology): “Thanks to them, the classification accuracy has been drastically increased [27, 72, 68, 35].”
- Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners (2023, background): “Many previous works have been proposed in this field to enhance model’s generalization capability by meta learning [20, 71], metric learning [74], and data augmentation [31, 73].”
- How to Train Your MAML to Excel in Few-Shot Classification (2021, methodology): “During meta-testing, we follow the evaluation protocol in [79, 110, 114] to sample 10, 000 tasks.”
- Fine-Grained Few-Shot Classification with Feature Map Reconstruction Networks (2020, methodology): “dard data augmentation as in [30, 31, 27, 3], which includes random crop, right-left flip and color jitter.”
- Few-Shot Learning With a Strong Teacher (2021, methodology): “Some recent methods emphasize the importance of pretrained features [44], [64], [23], [19], [34], [35].”
- Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation (2021, background): “DeepEMD [41] proposed the structural distance between dense image representations.”
- Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation (2020, methodology): “The conventional metric models in metric learning are often simple and have an explicit mathematical definition, e.g., Euclidean distance, Cosine distance and Earth Mover’s distance [11–13].”