Hard-Aware Deeply Cascaded Embedding

By Yuhui Yuan, Kuiyuan Yang. 2016

314 citations recorded by the source corpus.

Riding on the waves of deep neural networks, deep metric learning has achieved promising results in various tasks by using triplet network or Siamese network. Though the basic goal of making images from the same category closer than the ones from different categories is intuitive, it is hard to optimize the objective directly due to the quadratic or cubic sample size. Hard example mining is widely used to solve the problem, which spends the expensive computation on a subset of samples that are considered hard. However, hard is defined relative to a specific model. Then complex models will treat most samples as easy ones and vice versa for simple models, both of which are not good for training. It is difficult to define a model with the just right complexity and choose hard examples adequately as different samples are of diverse hard levels. This motivates us to propose the novel framework named Hard-Aware Deeply Cascaded Embedding(HDC) to ensemble a set of models with different complexities in cascaded manner to mine hard examples at multiple levels. A sample is judged by a series of models with increasing complexities and only updates models that consider the sample as a hard case. The HDC is evaluated on CARS196, CUB-200-2011, Stanford Online Products, VehicleID and DeepFashion datasets, and outperforms state-of-the-art methods by a large margin.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Why Should We Add Early Exits to Neural Networks? (2020, methodology): “on, similar approaches have been successfully applied to, among others, face recognition [66], object detection [15], image segmentation [16], pose estimation [27],saliencydetection[43],metriclearning[76],visualattentionprediction[74],andsuper resolution [36,70]. JT approaches are also popular in th”
Sampling Matters in Deep Embedding Learning (2017, methodology): “This is much more concise and simpler than HDC [41], which uses 3 embedding vectors for each image.”
Deep Metric Learning with Angular Loss (2017, methodology): “Second, it is beneﬁcial to combine our method with other practical tricks such as hard negative mining [37] or new clustering-like frameworks [21, 27].”
Attribute-Aware Attention Model for Fine-grained Representation Learning (2018, methodology): “s to learn a more discriminative attribute classification model. 4.5 Evaluations on Image Retrieval To ensure the fairness of the comparison, we reproduce the HDC method based on the InceptionBN. HDC [47] trains an ensemble of diverse models of different hardness levels, achieving Recall@1 at 60.0% ”
Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization (2020, methodology): “scrimination [17,44]. Recently, the metric learning strategies have attracted a lot of attention on EISNet 5 face verication and recognition [36], ne-grained object recognition [44], image retrieval [48], and so on. Dierent from previous applications, in this work, we adopt the conventional triplet ”
Proxy Anchor Loss for Deep Metric Learning (2020, background): “85.3 90.5 94.2 SoftTriple64 [23] BN 60.1 71.9 81.2 88.5 78.6 86.6 91.8 95.4 Proxy-Anchor64 BN 61.7 73.0 81.8 88.8 78.8 87.0 92.2 95.5 Margin128 [37] R50 63.6 74.4 83.1 90.0 79.6 86.5 91.9 95.1 HDC384 [40] G 53.6 65.7 77.0 85.6 73.7 83.2 89.5 93.8 A-BIER512 [22] G 57.5 68.7 78.3 86.2 82.0 89.0 93.2 9”
Imbalance Problems in Object Detection: A Review (2019, background): “Another approach that avoids considering the hardest possible set of examples is Hard-Aware Deeply Cascaded Embedding [167], which proposes training a cascaded model such that the higher layers are trained with harder examples while the first layers are trained with the entire dataset.”
Smart Mining for Deep Metric Learning (2017, background): “The development of deep metric learning models for the estimation of effective feature embedding [2, 4, 9, 11, 15, 16, 17, 13, 22, 25, 27, 26] is at the core of many recently proposed computer vision methods [3, 14, 19, 24, 28].”
Embedding-based Retrieval in Facebook Search (2020, result): “del focuses on recall and the second stage model specializes at differentiating more similar results returned by the first stage model. We shared the same spirit as the cascaded embedding training in [18], which ensembled a set of models trained with different level of hardness in a cascaded manner.”
Classification is a Strong Baseline for Deep Metric Learning (2018, methodology): “To focus on contributions from the loss functions, we leave comparisons against methods that ensemble models [29, 30], modify the feature extractor architecture [26], or propose complex activation paths between the featurizer and ﬁnal embedding [15] for Section 4.6.”
Heated-Up Softmax Embedding (2018, background): “2 73.0 66.6 77.5 256 67.3 72.2 69.7 78.0 75.2 70.1 80.1 8 Heated-Up Softmax Embedding Fashion dataset, we compared with [7] FashionNet (Liu et al., 2016) and [8] Hard-Aware Deeply Cascaded Embedding (Yuan et al., 2017). 4.3 METRIC LEARNING The performances of all the methods on all 4 datasets are li”
A Metric Learning Reality Check (2020, methodology): “s in a variety of negatives (easy, semihard, and hard), and improved performance. Online mining can also be integrated into the structure of models. Specically, the hard-aware deeply cascaded method [71] uses models of varying complexity, in which the loss for the complex models only considers the p”
VERI-Wild: A Large Dataset and a New Method for Vehicle Re-Identification in the Wild (2019, methodology): “HDC [28] also pays attention to the hard negatives, while they focus on mining hard negatives in the training set.”
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling (2019, methodology): “Methods R@1 R@2 R@4 R@8 NMI HDC [32] 53.6 65.7 77.0 85.6 - Margin [12] 63.6 74.4 83.1 90.0 69.0 HTL [3] 57.”
Learning an Evolutionary Embedding via Massive Knowledge Distillation (2020, background): “HDC [68] learns an ensemble to models with different complexities and finds hard examples adaptively.”
Deep Metric Learning with Hierarchical Triplet Loss (2018, methodology): “The proposed method achieves 80.9% Histogram Loss[30] 50.3 61.9 72.6 82.4 88.8 93.7 N-Pair-Loss[25] 51.0 63.3 74.3 83.2 - - HDC[38] 53.6 65.7 77.0 85.6 91.5 95.5 BIER[18] 55.3 Recall@1, and outperforms the baseline algorithm — triplet loss by 18.6%.”
Bi-Directional Cascade Network for Perceptual Edge Detection (2019, background): “[54] ensemble a set of models with different complexities to process samples with different difficulties.”
Conditional Negative Sampling for Contrastive Learning of Visual Representations (2020, background): “For instance, prior work in metric learning has shown the effectiveness of hard negative mining in optimizing triplet losses (Wu et al., 2017; Yuan et al., 2017; Schroff et al., 2015).”