Learning Fine-Grained Image Similarity with Deep Ranking

By Jiang Wang, Yang Song, Thomas K. Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Ying Wu. 2014

1,369 citations recorded by the source corpus.

Learning fine-grained image similarity is a challenging task. It needs to capture between-class and within-class image differences. This paper proposes a deep ranking model that employs deep learning techniques to learn similarity metric directly from images. It has higher learning capability than models based on hand-crafted features. A novel multiscale network structure has been developed to describe the images effectively. An efficient triplet sampling algorithm is also proposed to learn the model with distributed asynchronized stochastic gradient. Extensive experiments show that the proposed algorithm outperforms models based on hand-crafted visual features and deep classification models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Revisiting IM2GPS in the Deep Learning Era (2017, neutral): “[5, 19, 35].”
PlaNet - Photo Geolocation with Convolutional Neural Networks (2016, methodology): “We expect Euclidean embeddings trained for image retrieval using a triplet loss [48] to deliver even higher mAP.”
FaceNet: A unified embedding for face recognition and clustering (2015, background): “[18] for ranking images by semantic and visual similarity.”
Learning Deep Structure-Preserving Image-Text Embeddings (2015, background): “Finally, our work is related to deep similarity learning [4, 6, 7, 16, 18, 39, 47], though we are solving a cross-view, not a within-view, matching problem.”
Unsupervised Learning of Visual Representations Using Videos (2015, methodology): “Finally, our work is also related to metric learning via deep networks [51, 31, 4, 17, 15, 22, 54].”
Improving the Robustness of Deep Neural Networks via Stability Training (2016, methodology): “We use training data as in [13] to train the feature embeddings for near-duplicate detection and similar-image ranking.”
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, methodology): “To scale [64] to the millions of images considered in this work, we binarize the output for an efficient hashing-based exact search.”
Weakly Supervised Disentanglement with Guarantees (2019, background): “Although supervision via ranking features prominently in the metric learning literature (McFee & Lanckriet, 2010; Wang et al., 2014), our focus in this paper will be on rank pairing in the context of disentanglement guarantees.”
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, background): “However, since the data of old tasks is non-available and the number of new samples is few, a uniﬁed feature space is prone to overﬁt the new tasks, and the discriminability and generalization ability of the feature space easily degrades.”
Semantic Drift Compensation for Class-Incremental Learning (2020, background): “[39].”
SphereFace: Deep Hypersphere Embedding for Face Recognition (2017, background): “Most widely used loss functions for deep metric learning are contrastive loss [1, 3] and triplet loss [32, 22, 6], and both impose Euclidean margin to features.”
Deep Metric Learning via Lifted Structured Feature Embedding (2015, background): “Red edges in the illustration represent positive pairs (same class) and the blue edges represent negative pairs (different class) in the batch.”
Deep Metric Learning Using Triplet Network (2014, methodology): “Compared with the single application proposed in Wang et al. (2014), we make a comprehensive study of the triplet architecture which is, as we shall argue below, interesting in and of itself.”
Transitive Invariance for Self-Supervised Visual Representation Learning (2017, methodology): “We design a Triplet-Siamese network with a ranking loss function [54, 56] such that the distance between related samples should be smaller than the distance of unrelated samples.”
Compact Bilinear Pooling (2015, methodology): “used a 4096 dimensional feature embedding in their recent retrieval system [38].”
Bayesian representation learning with oracle constraints (2015, methodology): “Similar approaches have been used in Wang et al. (2014); Schroff et al. (2015), where usage of supervised features with crowd-inferred similarities boosts performance in face veriﬁcation and more generic ﬁne-grained visual categorization tasks.”
Joint Unsupervised Learning of Deep Representations and Image Clusters (2016, background): “Note that such triplet losses have appeared in other works [45, 54].”