Metric Learning with Adaptive Density Discrimination

By Oren Rippel, Manohar Paluri, Piotr Doll'ar, Lubomir D. Bourdev. 2015

224 citations recorded by the source corpus.

Distance metric learning (DML) approaches learn a transformation to a representation space where distance is in correspondence with a predefined notion of similarity. While such models offer a number of compelling benefits, it has been difficult for these to compete with modern classification algorithms in performance and even in feature extraction. In this work, we propose a novel approach explicitly designed to address a number of subtle yet important issues which have stymied earlier DML algorithms. It maintains an explicit model of the distributions of the different classes in representation space. It then employs this knowledge to adaptively assess similarity, and achieve local discrimination by penalizing class distribution overlap. We demonstrate the effectiveness of this idea on several tasks. Our approach achieves state-of-the-art classification results on a number of fine-grained visual recognition datasets, surpassing the standard softmax classifier and outperforming triplet loss by a relative margin of 30-40%. In terms of computational performance, it alleviates training inefficiencies in the traditional triplet loss, reaching the same error in 5-30 times fewer iterations. Beyond classification, we further validate the saliency of the learnt representations via their attribute concentration and hierarchy recovery properties, achieving 10-25% relative gains on the softmax classifier and 25-50% on triplet loss in these tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Revisiting IM2GPS in the Deep Learning Era (2017, methodology): “New loss functions and example mining strategies have been being proposed as they play important role in the learning process [24, 20, 23, 34].”
Grafit: Learning fine-grained image representations with coarse labels (2020, methodology): “[61], several Distance Metric Learning (DML) approaches like the Magnet loss [44] or ProxyNCA [40, 51] jointly take into account intra- and inter-class variability.”
Prototypical Networks for Few-shot Learning (2017, methodology): “[27]; however both methods require a separate partitioning phase that is decoupled from the weight updates, while our approach is simple to learn with ordinary gradient descent methods.”
Semantic Drift Compensation for Class-Incremental Learning (2020, background): “Embedding networks were found to be more robust to the exposure of adversarial examples, and better in the detection of out-of-distribution examples [25, 32].”
The Devil is in the Tails: Fine-grained Classification in the Wild (2017, background): “While these techniques appear promising and provide beneﬁts beyond classiﬁcation, they do not hold up well against simple baseline networks for the speciﬁc task of classiﬁcation [41].”
No Fuss Distance Metric Learning Using Proxies (2017, methodology): “In [14], the N-Pair Loss was proposed, which used Softmax cross-entropy loss on pairwise similarity values within the batch.”
A Comprehensive Study on Center Loss for Deep Face Recognition (2019, background): “Magnet loss (Rippel et al. 2015) is proposed to explicitly address the deep metric learning with the concept of center .”
Low-Shot Visual Recognition by Shrinking and Hallucinating Features (2016, methodology): “On benchmarks involving more general Internet imagery, such as ImageNet [9], these methods perform worse than simple classification baselines [34], and it is unclear if they can benefit low-shot learning.”
Low-shot visual object recognition (2016, methodology): “On more challenging benchmarks such as ImageNet [6], these methods perform worse than simple classification baselines [31], and it is unclear if they can benefit low-shot learning.”
Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces (2020, background): “…for the Triplet loss is very challenging as there is a combinatorial explosion in the number of triplets especially for large-scale datasets, requiring effective sampling strategies to select informative mini-batch [25,26] and choose representative triplets within the mini-batch [36,21,28].”
A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classification (2019, background): “In particular, several metric learning based methods (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017; Rippel et al., 2015) has been proposed for few-shot classification where a model first learns an embedding space and then a simple metric is used to classify instances of new…”
Central Similarity Quantization for Efficient Image and Video Retrieval (2019, background): “However, we empirically ﬁnd that learned centers by some common meth-ods [11, 43, 29] cannot provide better hash functions than the analytically constructed ones.”
One-Shot Learning in Discriminative Neural Networks (2017, methodology): “Recent work has looked at ways at improving the metric, such as triplet loss and magnet loss [Rippel et al., 2016] or deep nonlinear transformations which learn embeddings such that objects of the same class are near each other, such as siamese networks [Koch et al.”
ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis (2020, methodology): “One method that falls under this line of work is the Magnet Loss [19] in which samples are associated with a cluster centroid, and at each training batch, samples are attracted to cluster centroids of similar classes and repelled by cluster centroids of different classes.”
Hard-Aware Deeply Cascaded Embedding (2016, methodology): “[19] design a Nearest Class Multiple Centroids (NCMC) like loss which encourages images from the same category to form sub-clusters in the embedding space.”
Generative Adversarial Residual Pairwise Networks for One Shot Learning (2017, methodology): “Other related approaches include those based on Deep Metric Learning such as deep embeddings for triplet loss [13] and better metrics for fine-grained similarity [24], and popular methods like Neighbourhood Component Analysis [15].”
Clustering-Oriented Representation Learning with Attractive-Repulsive Loss (2018, methodology): “Recent work (Rippel et al. 2016) has modelled classes with a set of Gaussians with the motivation of creating a well-structured latent space, using neighborhood-based sampling to maintain the centers of these Gaussians.”
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling (2019, background): “Many efforts have been devoted to studying sampling an informative mini-batch [19, 21] and sampling triplets within a mini-batch [12, 24].”