A Survey on Metric Learning for Feature Vectors and Structured Data

By Aurélien Bellet, Amaury Habrard, Marc Sebban. 2013

714 citations recorded by the source corpus.

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2018, background): “Finally, there are many use cases for metric learning; see [54] or [7] for example.”
Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization (2020, methodology): “…methods that focus on preserving local structure are t-SNE (van der Maaten and Hinton, 2008), LargeVis (Tang et al., 2016), and UMAP (McInnes et al., 2018) (see van der Maaten et al., 2009; Bellet et al., 2013; Belkina et al., 2019, for a survey of these methods and how to tune their parameters).”
Deep Weighted Averaging Classifiers (2018, methodology): “eights are a function of the distance from the instance to be evaluated to all training instances, in the low-dimensional space. This is closely related to a long line of past work on metric learning [3, 10, 23, 58, 60], but rather than trying to optimize a particular notion of distance (such as Mah”
A Theoretical Analysis of Contrastive Unsupervised Representation Learning (2019, background): “Though reminiscent of past ideas – e.g. kernel learning, metric learning, co-training (Cortes et al., 2010; Bellet et al., 2013; Blum & Mitchell, 1998) – these algorithms lack a theoretical framework quantifying when and why they work.”
Few-Shot Learning Through an Information Retrieval Lens (2017, neutral): “We refer the reader to a survey of metric learning [5] and merely touch upon a few representative methods here.”
Task-Embedded Control Networks for Few-Shot Imitation Learning (2018, background): “The success of our new approach comes from learning a metric space, and there has been an abundance of work in metric learning for image classification [23, 24], from which we will summarise the most relevant.”
Decomposing fit semantics for product size recommendation in metric spaces (2018, methodology): “However, this also makes the method very sensitive to the ability of the Euclidean distance to select relevant target neighbors [2].”
A Theoretical Analysis of Metric Hypothesis Transfer Learning (2015, background): “This has led to the emergence of supervised metric learning, see (Bellet et al., 2013; Kulis, 2013) for up-to-date surveys.”
Deep Metric Learning: A Survey (2019, background): “These approaches have some advantages, such as convex formulations and robustness to overfitting [16].”
Riemannian Approaches in Brain-Computer Interfaces: A Review (2017, neutral): “As reviewed in [51], it has been successfully applied to a great variety of data.”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “The distance metric learning has been also used very extensively for feature vectors representation [20].”
Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication (2016, background): “One usual way to do so is to express the problem as a binary classiﬁcation on pairs of samples, where the role of the metric is to separate similar and dissimilar samples by thresholding (see [42] for a survey on M.L.).”
Lifelong Metric Learning (2017, background): “Furthermore, more details can also be found in two surveys [27] and [28].”
Dynamic Input Structure and Network Assembly for Few-Shot Learning (2017, methodology): “The second stage of the network focuses on learning an internal distance metric, similar to metric learning as in Bellet et al. (2013).”