No Fuss Distance Metric Learning Using Proxies

By Yair Movshovitz-Attias, Alexander Toshev, Thomas K. Leung, Sergey Ioffe, Saurabh Singh. 2017

713 citations recorded by the source corpus.

We address the problem of distance metric learning (DML), defined as learning a distance consistent with a notion of semantic similarity. Traditionally, for this problem supervision is expressed in the form of sets of points that follow an ordinal relationship – an anchor point x is similar to a set of positive points Y , and dissimilar to a set of negative points Z, and a loss defined over these distances is minimized. While the specifics of the optimization differ, in this work we collectively call this type of supervision Triplets and all methods that follow this pattern Triplet-Based methods. These methods are challenging to optimize. A main issue is the need for finding informative triplets, which is usually achieved by a variety of tricks such as increasing the batch size, hard or semi-hard triplet mining, etc. Even with these tricks, the convergence rate of such methods is slow. In this paper we propose to optimize the triplet loss on a different space of triplets, consisting of an anchor data point and similar and dissimilar proxy points which are learned as well. These proxies approximate the original data points, so that a triplet loss over the proxies is a tight upper bound of the original loss. This proxy-based loss is empirically better behaved. As a result, the proxy-loss improves on state-of-art results for three standard zero-shot learning datasets, by up to 15% points, while converging three times as fast as other triplet-based losses.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Representation & Self-Supervised Learning

Citations in the corpus

Grafit: Learning fine-grained image representations with coarse labels (2020, methodology): “[61], several Distance Metric Learning (DML) approaches like the Magnet loss [44] or ProxyNCA [40, 51] jointly take into account intra- and inter-class variability.”
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature (2019, methodology): “Su-pervised embedding learning methods have been studied to achieve such objectives and demonstrate impressive capabilities in various vision tasks [28, 30, 53].”
Learning Spread-Out Local Feature Descriptors (2017, background): “Structured loss [12, 13, 14, 21, 23] considers all the possible matching and non-matching pairs in one batch of samples.”
Class-Incremental Learning for Action Recognition in Videos (2021, positive): “For the classiﬁcation loss L k cls , we adopt NCA loss [25] computed from the Local Similarity Classiﬁer (LSC) following [6].”
A Baseline for Few-Shot Image Classification (2019, methodology): “The support-based initialization is motivated from previous papers [7, 37, 38, 39].”
Mining on Manifolds: Metric Learning Without Labels (2018, background): “Our goal is to mine such training pairs without any supervision [9, 62, 18, 18, 34], without any complementary computer vision system [15, 42] or assumptions on the nature of the training data [64, 65, 11].”
Classification is a Strong Baseline for Deep Metric Learning (2018, methodology): “The theoretical link between classiﬁcation and metric learning is shown in [14], and image retrieval tasks have some success adopting classiﬁcation loss [14, 29].”
Sub-center ArcFace: Boosting Face Recognition by Large-Scale Noisy Web Faces (2020, background): “Some works tried to reduce the total number of triplets with proxies [19,23], i.e., sample-to-sample comparison is changed into sample-to-proxy comparison.”
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning (2020, background): “[21] leverage a contrastive loss to enforce the positive samples to stay in a close neighborhood and negative samples far apart [26, 27, 28, 29].”
ABO: Dataset and Benchmarks for Real-World 3D Object Understanding (2021, background): “Designs of losses for representation and metric learning in the recent literature [43] can be broadly summarized into: (i) pair-based losses [22, 44], where losses are computed based on 2, 3 or more distances between samples in the batches, with the goal to minimize the distances between the sample ”
Learning With Average Precision: Training Image Retrieval With a Listwise Loss (2019, background): “However, several works [15, 25, 28, 40, 48] pointed out that properly optimizing a local loss can be a challenging task, for several reasons.”
SphereFace Revived: Unifying Hyperspherical Face Recognition (2021, background): “The development of deep FR is also closely related to deep metric learning [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28].”
ContraGAN: Contrastive Learning for Conditional Image Generation (2020, methodology): “GAN [6] with P-NCA loss [33] with NT-Xent loss [34] Our 2C loss can take advantage of both losses.”
Deep Metric Learning: A Survey (2019, methodology): “For this reason, to overcome these problems, it is very convenient to use informative sample triplets, and more feasible train models with a better sample strategy could be provided instead of selecting random samples [91,92].”
Learning with Imprinted Weights (2017, methodology): “Note that storing all embeddings trained with the softmax loss in a nearest-neighbor classiﬁer is equivalent to a special case of Proxy-NCA [10] using one proxy per class.”
Composing Text and Image for Image Retrieval - an Empirical Odyssey (2018, methodology): “If we use larger K, each example is contrasted with a set of other negatives; this loss resembles the classification based loss used in [9, 46, 32, 45, 8].”