Deep semantic ranking based hashing for multi-label image retrieval

By Fang Zhao, Yongzhen Huang, Liang Wang, Tien-Ping Tan. Computer Vision and Pattern Recognition, 2015

632 citations recorded by the source corpus.

With the rapid growth of web images, hashing has received increasing interests in large scale image retrieval. Research efforts have been devoted to learning compact binary codes that preserve semantic similarity based on labels. However, most of these hashing methods are designed to handle simple binary similarity. The complex multi-level semantic structure of images associated with multiple labels have not yet been well explored. Here we propose a deep semantic ranking based method for learning hash functions that preserve multilevel semantic similarity between multi-label images. In our approach, deep convolutional neural network is incorporated into hash functions to jointly learn feature representations and mappings from them to hash codes, which avoids the limitation of semantic representation power of hand-crafted features. Meanwhile, a ranking list that encodes the multilevel similarity information is employed to guide the learning of such deep hash functions. An effective scheme based on surrogate loss is used to solve the intractable optimization problem of nonsmooth and multivariate ranking measures involved in the learning procedure. Experimental results show the superiority of our proposed approach over several state-of-the-art hashing methods in term of ranking evaluation metrics when tested on multi-label image datasets.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b48c2bccc3ae0d4fc0839ac0ffdc8eefa100ac7c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Bit-Scalable Deep Hashing With Regularized Similarity Learning for Image Retrieval and Person Re-Identification (2015, methodology): “Note that the network parameter s of DSRH [40] and our method are initialized randomly without any pre-training.”
Deep Cross-Modal Hashing (2016, methodology): “There also exist some methods which adopt deep learning for uni-modal hashing (Zhao et al., 2015; Liong et al., 2015).”
Transductive Semi-Supervised Deep Learning Using Min-Max Features (2018, background): “To date, Deep Convolutional Neural Networks (DCNNs) have shown state-of-the-art performances in numerous computer vision applications, such as image classiﬁcation [1–5], object detection [6, 7], face recognition [8–10], image retrieval [11–14], etc.”
Deep Supervised Discrete Hashing (2017, methodology): “Deep Semantic Ranking Hashing (DSRH) [24] learns the hash functions by preserving semantic similarity between multi-label images.”
Unsupervised Semantic-Preserving Adversarial Hashing for Image Search (2019, methodology): “We further compare UADH with several popular supervised hashing methods, including DRSCH [70], DSCH [70], DSRH [71], and DPSH [41].”
SIFT Meets CNN: A Decade Survey of Instance Retrieval (2016, methodology): “These methods take the original image as input and produce a learned feature before binarization [159], [160].”
Person Re-identification: Past, Present and Future (2016, background): “Another application of supervised hashing consists of image retrieval [152], [153], [154], [155], which is the interest of this section.”
HashNet: Deep Learning to Hash by Continuation (2017, background): “nce on many benchmarks. In particular, it proves crucial to jointly learn similarity-preserving representations and control quantization error of binarizing continuous representations to binary codes [44,22,43,24]. However, a key disadvantage of these deep learning to hash methods is that they need ”
Unsupervised Learning of Discriminative Attributes and Visual Representations (2016, methodology): “We share the same merits, but differ in two aspects: 1) Our method is label-free unlike the supervised ones [27, 31, 51, 52].”
Learning to Hash for Indexing Big Data—A Survey (2015, methodology): “Similar to the deep semantic ranking hashing method [108], the deep neural network hashing method incorporates listwise supervised information to train a deep CNN, giving rise to a currently deepest architecture for supervised hashing.”
Two-Stage Label Embedding via Neural Factorization Machine for Multi-Label Classification (2019, result): “In contrast to other neural network based models (Yeh et al. 2017; Zhao et al. 2015) which learn high-order correlations directly, TSLE uses the factorization layer to extract second-order interactions between features and labels in advance.”
Deep Supervised Hashing for Fast Image Retrieval (2016, methodology): “To tackle this issue, most recently, several CNNbased hashing methods [31, 14, 29, 15, 30] are proposed to learn image representations together with binary codes using the promising CNNs.”
How should we evaluate supervised hashing? (2016, methodology): “We distinguish three classes of methods for supervised hashing: triplet loss hashing [1, 2, 3, 4], pairwise similarity-based and label-based.”
A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “have utilized the multilevel similarity information using deep semantic ranking for image retrieval [210].”
Triplet-Based Deep Hashing Network for Cross-Modal Retrieval (2018, methodology): “Although [27]–[31] also employ triplet labels, they apply then only to image retrieval whose query data and retrieval results are from the same modalities; Unlike these studies, our proposed method extends triplet labels to cross-modal retrieval whose query data and retrieval results are from the di”
Feature Learning Based Deep Supervised Hashing with Pairwise Labels (2015, methodology): “• Deep hashing methods with triplet labels, including network in network hashing (NINH) [10], deep semantic ranking based hashing (DSRH) [37], deep similarity comparison hashing (DSCH) [36] and deep regularized similarity comparison hashing (DRSCH) [36].”
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval (2018, methodology): “In the initial attempt, feature representation and hashing codes were learned in separate stages [28], where subsequent work [14, 35, 15] suggested superior practice through joint end-to-end training.”
Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification (2017, methodology): “The motivation is based on the evidence that in detection, segmentation and retrieval, they have used the pretrained classification model (on ImageNet) as initialization to boost performance[19, 4, 31].”