Triplet-Based Deep Hashing Network for Cross-Modal Retrieval

By Cheng Deng, Zhaojia Chen, Xianglong Liu, Xinbo Gao, Dacheng Tao. IEEE Transactions on Image Processing, 2018

371 citations recorded by the source corpus.

Given the benefits of its low storage requirements and high retrieval efficiency, hashing has recently received increasing attention. In particular, cross-modal hashing has been widely and successfully used in multimedia similarity search applications. However, almost all existing methods employing cross-modal hashing cannot obtain powerful hash codes due to their ignoring the relative similarity between heterogeneous data that contains richer semantic information, leading to unsatisfactory retrieval performance. In this paper, we propose a triplet-based deep hashing (TDH) network for cross-modal retrieval. First, we utilize the triplet labels, which describe the relative relationships among three instances as supervision in order to capture more general semantic correlations between cross-modal instances. We then establish a loss function from the inter-modal view and the intra-modal view to boost the discriminative abilities of the hash codes. Finally, graph regularization is introduced into our proposed TDH method to preserve the original semantic similarity between hash codes in Hamming space. Experimental results show that our proposed method outperforms several state-of-the-art approaches on two popular cross-modal data sets.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2c8c7c9f4e8f158e202ee02f562aedd6b434bfff

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Person Re-identification (2017, background): “Despite the impressive progress achieved by the deep learning methods [3], [10], [8], most of these methods focus on re-ID with sufﬁcient training data in the same environment.”
Two-Stream Deep Hashing With Class-Specific Centers for Supervised Image Search (2020, background): “Deep hashing (DH) methods usually learn hash codes by ﬁrst constructing pairwise [43] or triplet [18] similarity relationships, and then learn hash functions to generate hash codes from images that are consistent with these relationships.”
Unsupervised Semantic-Preserving Adversarial Hashing for Image Search (2019, methodology): “Usually, these methods are supervised by massive numbers of ranking pairwise [41] or triplet labels [43].”
COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (2021, background): “As one of most effective unsupervised learning paradigms, contrastive learning [2, 4, 8, 23, 28, 30, 37, 38] has achieved state-of-the-art performance in representation learning.”
Deep Multimodal Data Fusion (2024, methodology): “…related to graph data, e.g., VQA task [77, 95, 100, 129, 190, 214, 218], image captioning task [42, 182, 182, 208–210], cross-modal retrieval task [30, 204, 217], RGB-depth scene classification task [222], multimodal recommendation task [193, 194], neuroimaging-based disease classification task…”
Scalable Deep Multimodal Learning for Cross-Modal Retrieval (2019, background): “Furthermore, some works attempt to project multi-modal data into a latent common subspace by metric learning [20], cross-modal translation [30], and cross-modal hashing [5].”
Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing (2020, methodology): “From this perspective, the supervised CMH methods [1, 25, 22, 34, 44], receiving additional supervision, often produce more accurate results, and the unsupervised counterparts, while achieving lower performance, are relatively easier to deploy to real-world scenarios.”
Deep multi-view learning methods: A review (2021, methodology): “For the deep cross-modal retrieval methods, we compare the mean average precision (mAP) [162] of the following approaches: deep joint-semantics reconstructing hashing (DJSRH) [90], self-supervised 655 adversarial hashing (SSAH) [160], unsupervised deep cross-modal hashing (UDCMH) [159], unsupervised”
Trusted Multi-View Classification With Dynamic Evidential Fusion (2022, methodology): “These methods are typically built on complex models [1], [2], [3], [4], [5], [6], [7], [8], [9] (usually integrating different views with deep neural networks (DNNs) ) and rely on large datasets with semantic annotations.”
University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization (2020, methodology): “ge datasets with the viewpoint-invariant feature learned on the proposed dataset. We also evaluate three basic models and three different loss terms, including contrastive loss [14, 31], triplet loss [4, 5], and instance loss [36]. Apart from the extensive evaluation of the baseline method, we also ”
Complementary Incremental Hashing With Query-Adaptive Re-Ranking for Image Retrieval (2020, background): “Triplet-based deep hashing [32] learns discriminative hash codes based on the relative similarity information between heterogeneous data for cross-modal retrieval.”
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining (2021, background): “Cross-modal retrieval [43, 12, 48, 4, 44, 8] emerges as a promising avenue for efficient indexing and searching among large-scale data with different modalities, and is widely used in search engines [2, 14], E-commerce [18, 7], to name a few.”
Adversarial Fine-Grained Composition Learning for Unseen Attribute-Object Recognition (2019, methodology): “Besides, other methods [19, 20] attempt to model the diverse compositions of attributes and objects, and project the compositions as well as the image visual features to a common embedding space with regularization, such as triplet loss [11, 27, 6].”
Two-Stage Label Embedding via Neural Factorization Machine for Multi-Label Classification (2019, methodology): “Embedding method (Hsu et al. 2009; Cao et al. 2016; Yang et al. 2018; Deng et al. 2018a) is one of the most popular frameworks to learn label and feature correlations and has shown promising results.”
Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization (2020, methodology): “The performance has surpassed the reported result of other competitive methods such as [21], [20], [54], [55], [7], [4], and the proposed method outperforms the best method, i.”