Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval

By Xing Xu, Fumin Shen, Yang Yang, Heng Tao Shen, Xuelong Li. IEEE Transactions on Image Processing, 2017

427 citations recorded by the source corpus.

Hashing based methods have attracted considerable attention for efficient cross-modal retrieval on large-scale multimedia data. The core problem of cross-modal hashing is how to learn compact binary codes that construct the underlying correlations between heterogeneous features from different modalities. A majority of recent approaches aim at learning hash functions to preserve the pairwise similarities defined by given class labels. However, these methods fail to explicitly explore the discriminative property of class labels during hash function learning. In addition, they usually discard the discrete constraints imposed on the to-be-learned binary codes, and compromise to solve a relaxed problem with quantization to obtain the approximate binary solution. Therefore, the binary codes generated by these methods are suboptimal and less discriminative to different classes. To overcome these drawbacks, we propose a novel cross-modal hashing method, termed discrete cross-modal hashing (DCH), which directly learns discriminative binary codes while retaining the discrete constraints. Specifically, DCH learns modality-specific hash functions for generating unified binary codes, and these binary codes are viewed as representative features for discriminative classification with class labels. An effective discrete optimization algorithm is developed for DCH to jointly learn the modality-specific hash function and the unified binary codes. Extensive experiments on three benchmark data sets highlight the superiority of DCH under various cross-modal scenarios and show its state-of-the-art performance.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e397b9cad491b76a2240fb7731c40cde4c2a2f7c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Shiv Ram Dubey
3. Heng Tao Shen
4. Jingkuan Song
5. Cheng Deng
6. Li Liu
7. Jingdong Wang
8. Yi Yang

Citations in the corpus

• Adversarial Cross-Modal Retrieval (2017, methodology): “The binary approaches, also referred to as cross-modal hashing are more geared towards retrieval efficiency and aim at mapping the items of different modalities into a common binary Hamming space [24, 27, 34, 42].”
• Triplet-Based Deep Hashing Network for Cross-Modal Retrieval (2018, methodology): “In the development of deep neural networks (DNN), deep models are applied to crossmodal hashing [17] [24]–[26] to efficiently capture the correlations between heterogeneous data.”
• Deep Multi-View Enhancement Hashing for Image Retrieval (2020, background): “Recently, an increasing number of researches explicitly or implicitly capture the relations among different views [19], [20] and modalities [21], [22] to enhance the multi/cross-view/modal approaches.”
• The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers (2018, methodology): “For instance, SIFT [71], [72] and other local descriptors [73], [74], [75] have been intensively used in the domain of object recognition and image retrieval [76], [77], [78] for decades as”
• Unsupervised Deep Hashing with Similarity-Adaptive and Discrete Optimization (2018, methodology): “Many learning-based hashing algorithms have been developed according to different scenarios, including the supervised methods [22], [30], [37], [40], [47], unsupervised methods [10], [12], [29], [31], [49] and cross-modal hashing methods [17], [24], [53], [58], etc.”
• Self-Supervised Learning from Web Data for Multimodal Retrieval (2019, methodology): “Following the experimental protocol in [35, 36, 37, 38] tags that appear less than 20 times are first removed and then instances without tags or annotations are removed.”
• High-Dimensional Sparse Cross-Modal Hashing with Fine-Grained Similarity Embedding (2021, background): “, DCH [54], SCRATCH [3], LCMFH [46], and DLFH [20].”
• A Survey on Learning to Hash (2016, background): “This topic is attracting a lot of research efforts nowadays, such as collaborative hashing [89], [167], collaborative quantization [173], and cross-media hashing [134], [135], [178], [161], [83].”