Dimensionality Reduction by Learning an Invariant Mapping

By Raia Hadsell, Sumit Chopra, Yann LeCun. 2006

5,839 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

In Representation & Self-Supervised Learning

Citations in the corpus

Masked Autoencoders Are Scalable Vision Learners (2021, background): “Recently, contrastive learning [3, 22] has been popular, e.”
A Simple Framework for Contrastive Learning of Visual Representations (2020, positive): “Discriminative approaches based on contrastive learning in the latent space have recently shown great promise, achieving state-of-theart results (Hadsell et al., 2006; Dosovitskiy et al., 2014; Oord et al., 2018; Bachman et al., 2019).”
Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “Learning is formulated as minimizing a contrastive loss [29]. representation encoder by matching an encoded query q to a dictionary of encoded keys using a contrastive loss.”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “These predictions are evaluated using a contrastive loss (Chopra et al., 2005; Hadsell et al., 2006), in which the network must correctly classify ‘future’ representations among a set of unrelated ‘negative’ representations.”
VirTex: Learning Visual Representations from Textual Annotations (2020, background): “Recent works are based on contrastive learning [49, 50], encouraging similarity between image features under different random transformations on single input image [24–26, 51, 52].”
Improved Baselines with Momentum Contrastive Learning (2020, background): “Contrastive learning [5] is a framework that learns similar/dissimilar representations from data that are organized into similar/dissimilar pairs.”
Contrastive Multiview Coding (2019, background): “[26, 81, 72, 25, 31, 57, 3, 29, 36] that take advantage of the co-occurence of multiple views.”
Revisiting IM2GPS in the Deep Learning Era (2017, neutral): “Distance metric learning (DML) is usually employed with a deep network, most commonly us-ing the contrastive loss [8] or triplet ranking (hinge loss) Figure 3.”
An Empirical Study of Training Self-Supervised Vision Transformers (2021, background): “In computer vision, contrastive learning [18] has become increasingly successful for self-supervised learning, e.”
Self-supervised Pretraining of Visual Features in the Wild (2021, methodology): “Selfsupervised learning has a long history in computer vision with methods based on autoencoders [42, 51], clustering [1, 5, 11] or instance-level discrimination [3, 15, 21, 52].”
Exploring Simple Siamese Representation Learning (2020, background): “The core idea of contrastive learning [16] is to attract the positive sample pairs and repulse the negative sample pairs.”
Dual Contradistinctive Generative Autoencoder (2020, neutral): “Inspired by the literature on instance-level classiﬁcation [50], approximating likelihood by classiﬁcation [60], and contrastive learning [20, 65, 22], we propose to model instance-level ﬁdelity by contrastive loss (commonly referred to as InfoNCE loss) [62].”
Self-Supervised Learning of Pretext-Invariant Representations (2019, methodology): “We implement `inv(·) using a contrastive loss function L(·, ·) [24].”
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature (2019, background): “Most of them are designed on top of pairwise [12, 30] or triplet relationships [13, 29].”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, neutral): “One can also use the temporal structure in a video for self-supervised methods [21, 27, 41, 46, 47].”
Generating Videos with Scene Dynamics (2016, methodology): “…STIP features [35], the network initialized with our model signiﬁcantly Method Accuracy Chance 0.9% STIP Features [35] 43.9% Temporal Coherence [10] 45.4% Shufﬂe and Learn [24] 50.2% VGAN + Random Init 36.7% VGAN + Logistic Reg 49.3% VGAN + Fine Tune 52.1% ImageNet Supervision [46]91.4% We…”
Deep Learning Face Attributes in the Wild (2014, methodology): “To improve intra-class invariance, we employ the similarity loss similar to [26, 10].”
Learning to See by Moving (2015, methodology): “For this purpose, we used image and odometry data recorded from a car moving through urban scenes, made available as part of the KITTI [15] and the San Francisco (SF) city [10] datasets.”
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (2020, neutral): “Recent self-supervised methods that use instance discrimination rely on a combination of two elements: (i) a contrastive loss [19] and (ii) a set of image transformations.”
Learning Image Representations Tied to Ego-Motion (2015, methodology): “90 DRLIM [9] 0.”