Distance Metric Learning for Large Margin Nearest Neighbor Classification

By Kilian Q. Weinberger, Lawrence K. Saul. 2005

5,993 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Matching Networks for One Shot Learning (2016, methodology): “This kind of loss is also related to methods such as Neighborhood Component Analysis (NCA) [18], triplet loss [9] or large margin nearest neighbor [28].”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “For example, camera motion [1, 32], tracking [61], text proximity [41], and other cues [11, 62, 30, 60, 29] can be used to ﬁnd examples of features that should be ‘similar’ or predictable from one another, whereas negative examples are usually sampled randomly.”
Representation Learning with Contrastive Predictive Coding (2018, methodology): “For example, the techniques proposed by [21, 22, 23] were based on triplet losses using a max-margin approach to separate positive from negative examples.”
FaceNet: A unified embedding for face recognition and clustering (2015, methodology): “D embedding using a tripletbased loss function based on LMNN [19].”
Unsupervised Pretraining Transfers Well Across Languages (2020, background): “for speech and we focus on those based on contrastive learning [7, 14, 15].”
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech (2019, background): “This loss function blends two triplet loss terms (Weinberger & Saul, 2009), one based on random sampling of negative examples, and the other based on semi-hard negative mining (Jansen et al., 2018), in order to find more challenging negative samples.”
Prototypical Networks for Few-shot Learning (2017, background): “Large margin nearest neighbor (LMNN) classification [33] also attempts to optimize KNN accuracy but does so using a hinge loss that encourages the local neighborhood of a point to contain other points with the same label.”
Learning Deep Structure-Preserving Image-Text Embeddings (2015, background): “Such within-view structure preservation constraints have been extensively explored in the metric learning literature [19, 32, 40, 41, 48, 53].”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “Since learning a projection for the data is equivalent to learning a Mahalanobis metric in the original feature space, our work is related to metric learning (Globerson et al., 2005; Goldberg et al., 2004; Weinberger et al., 2005).”
Deep Weighted Averaging Classifiers (2018, methodology): “This is closely related to a long line of past work on metric learning [3, 10, 23, 58, 60], but rather than trying to optimize a particular notion of distance (such as Mahalanobis distance), we make use of a fixed distance function, and allow the architecture of standard deep models to do the equiva”
This looks like that: deep learning for interpretable image recognition (2018, methodology): “Our work relates closely to other case-based classification techniques using k-nearest neighbors [47, 35, 32] or prototypes [33, 2, 48], and very closely, to the Bayesian Case Model [18].”
Set2Graph: Learning Graphs From Sets (2020, methodology): “For the particle physics application we also used: (2) The same architecture as (1) but with a triplet loss (Weinberger et al., 2006) on the learned representations based on l 2 distance; we name this base-line (TRI).”
SphereFace: Deep Hypersphere Embedding for Face Recognition (2017, background): “Traditional metric learning [36, 33, 12, 38] usually learns a matrix A for a distance metric ‖x1−x2‖A= √ (x1−x2)A(x1−x2) upon the given features x1,x2.”
Deep Metric Learning via Lifted Structured Feature Embedding (2015, background): “We split the first 100 classes for training (5,864 images) and the rest of the classes for testing (5,924 images).”
Supervised Contrastive Learning (2020, neutral): “Supervised contrastive learning is closely related to the triplet loss [52], one of the widely-used loss functions for supervised learning.”
Learning Two-Branch Neural Networks for Image-Text Matching Tasks (2017, background): “In particular, the Large Margin Nearest Neighbor (LMNN) approach [42] tries to ensure that for each image its target neighbors from the same class are closer than samples from other classes.”