Deep Cross-Modal Hashing

By Qing-Yuan Jiang, Wu-Jun Li. Computer Vision and Pattern Recognition, 2016

752 citations recorded by the source corpus.

Due to its low storage cost and fast query speed, cross-modal hashing (CMH) has been widely used for similarity search in multimedia retrieval applications. However, most existing CMH methods are based on hand-crafted features which might not be optimally compatible with the hash-code learning procedure. As a result, existing CMH methods with hand-crafted features may not achieve satisfactory performance. In this paper, we propose a novel CMH method, called deep cross-modal hashing (DCMH), by integrating feature learning and hash-code learning intothe same framework. DCMH is an end-to-end learning framework with deep neural networks, one for each modality, to perform feature learning from scratch. Experiments on three real datasets with image-text modalities show that DCMH can outperform other baselines to achieve the state-of-the-art performance in cross-modal retrieval applications.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2e34d2483808d6f0c78289fafc48c61dcdc068eb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Ling Shao
4. Qi Tian
5. Yi Yang
6. Alan L. Yuille
7. Xiaogang Wang
8. Tao A. Xiang

Citations in the corpus

• Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval (2018, methodology): “In comparison to DCMH, SSAH exploits LabNet to learn more sufficient supervised information from high-dimensional semantic features and hash codes, using it to train ImgNet and TxtNet efficiently.”
• A Comprehensive Survey on Cross-modal Retrieval (2016, methodology): “SePH [81], CAH [82], DCMH [83] in which cross-modal similarity search is fast.”
• Shared Predictive Cross-Modal Deep Quantization (2018, positive): “Besides, as shown in Table II, the proposed SPDQ outperforms all shallow baselines using CNN features with different code lengths on all retrieval tasks, and also outperforms DCMH, the state-of-the-art deep cross-modal hashing method, as well as CDQ the state-of-the-art deep cross-modal quantization”
• Deep Supervised Cross-Modal Retrieval (2019, methodology): “The recently proposed methods in [9], [22] and [30] also exploited semantic category labels to learn discriminative features for cross-modal retrieval.”
• Zero-Shot Sketch-Image Hashing (2018, methodology): “As an extension of conventional data hashing techniques [51, 20, 39, 54, 53], cross-modal hashing [4, 13, 72, 36, 33, 26, 5, 6] show great potential in retrieving heterogeneous data with high efficiency due to the computationally cheap Hamming space matching, which is recently adopted to large-scale”
• Deep Bayesian Hashing with Center Prior for Multi-modal Neuroimage Retrieval (2020, methodology): “Deep learning-based cross-modal hashing methods [26], [37]–[39] have also been proposed for image retrieval.”
• Deep Supervised Hashing for Fast Image Retrieval (2016, methodology): “Specifically, as a representative family of such methods, many hashing algorithms (Gionis et al. 1999; Gong and Lazebnik 2011; Jiang and Li 2017; Kang et al. 2016; Kulis and Darrell 2009; Lai et al. 2015; Li et al. 2016; Lin et al. 2015a; Liu et al. 2017, 2014, 2012; Norouzi and Fleet 2011; Rastegar”
• SCH-GAN: Semi-Supervised Cross-Modal Hashing by Generative Adversarial Network (2018, methodology): “8 0.456 0.488 SCM orth [17] 0.389 0.376 0.368 0.360 0.388 0.372 0.360 0.353 SCM seq [17] 0.517 0.514 0.518 0.518 0.518 0.510 0.517 0.518 SePH [19] 0.701 0.712 0.719 0.726 0.642 0.653 0.657 0.662 DCMH [50] 0.631 0.653 0.653 0.671 0.702 0.695 0.694 0.693 SCH-GAN (Ours) 0.713 0.726 0.734 0.748 0.741 0.”
• Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing (2020, methodology): “Here we utilize the framework of DCMH but modify its architecture for higher accuracy.”
• A Survey of Multi-View Representation Learning (2016, neutral): “The underlying idea is to make the topic decided by the visual information to be consistent with the topic decided by the textual information, leading to the correct label assignment.”
• AGCN: Augmented Graph Convolutional Network for Lifelong Multi-Label Image Recognition (2022, neutral): “Following [26], we choose the 21 most frequent concepts from 81 classes of NUS-WIDE to construct the Split-WIDE, which has 144,858 examples for training and 41,146 examples for validation.”
• Robust Unsupervised Cross-modal Hashing for Multimedia Retrieval (2020, methodology): “Supervised CMH methods leverage available semantic affinities [2, 3, 14] constructed by labels or label information of training data [4, 5, 9, 12, 16, 17, 43, 48] to supervise the learning process.”
• Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval (2017, methodology): “, unsupervised methods [12], [14], [15], [23]–[25] and supervised methods [26]–[31].”
• Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval (2017, methodology): “Moreover, DSH is also compared with state-of-theart cross-modality hashing techniques: Collective Matrix Factorization Hashing (CMFH) [10], Cross-Modal SemiSupervised Hashing (CMSSH) [2], Cross-View Hashing (CVH) [26], Semantic Correlation Maximization (SCMSeq and SCM-Orth) [68], Semantics-Preservin”
• Multimodal Machine Learning: A Survey and Taxonomy (2017, methodology): “For example, Jiang and Li [92] introduced a method to learn such common binary space between sentence descriptions and corresponding images”
• Triplet-Based Deep Hashing Network for Cross-Modal Retrieval (2018, methodology): “From the MAP results, we find that the increasing in MAP for the text query image task is less than that for image query text task, and that even the MAP of TDH is inferior to that of DCMH, with 64 bits on the NUS-WIDE dataset.”