Pairwise Relationship Guided Deep Hashing for Cross-Modal Retrieval

By Erkun Yang, Cheng Deng, Wei David Liu, Xianglong Liu, Dacheng Tao, Xinbo Gao. AAAI Conference on Artificial Intelligence, 2017

336 citations recorded by the source corpus.

With benefits of low storage cost and fast query speed, cross-modal hashing has received considerable attention recently. However, almost all existing methods on cross-modal hashing cannot obtain powerful hash codes due to directly utilizing hand-crafted features or ignoring heterogeneous correlations across different modalities, which will greatly degrade the retrieval performance. In this paper, we propose a novel deep cross-modal hashing method to generate compact hash codes through an end-to-end deep learning architecture, which can effectively capture the intrinsic relationships between various modalities. Our architecture integrates different types of pairwise constraints to encourage the similarities of the hash codes from an intra-modal view and an inter-modal view, respectively. Moreover, additional decorrelation constraints are introduced to this architecture, thus enhancing the discriminative ability of each hash bit. Extensive experiments show that our proposed method yields state-of-the-art results on two cross-modal retrieval datasets.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ced433d6e002474bc9e391c47e8bd6367da42591

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Shiv Ram Dubey
3. Ling Shao
4. Cheng Deng
5. Luc Van Gool
6. Chang Xu
7. Tongliang Liu
8. Li Liu

Citations in the corpus

• Unsupervised Semantic-Preserving Adversarial Hashing for Image Search (2019, methodology): “Deep learning-based hashing methods [41], [42] have also been proposed for efficient ANN search, showing that deep networks can help in the more effective learning of hash codes.”
• Zero-Shot Sketch-Image Hashing (2018, background): “General cross-modal binary representation learning methods [4, 13, 72, 33, 58, 43, 36, 26, 6, 63, 18, 55, 66, 5, 42] target to map large-scale heterogeneous data with low computational cost.”
• Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval (2018, background): “However, only a few works have performed deep learning for cross-modal hashing [3, 9, 12, 31, 43], which can capture nonlinear correlations among cross-modal instances more effectively.”
• Artificial Intelligence and COVID-19: Deep Learning Approaches for Diagnosis and Treatment (2020, methodology): “In addition to image generation tasks, GANs have been introduced to tasks, such as video generation, visual tracking [70], domain adaption [71], hashing coding [72], and”
• Learning With Single-Teacher Multi-Student (2018, background): “Other MC applications refer to image classification (Guo 2017), text categorization (Xu et al. 2017), information retrieval (Yang et al. 2017), multi-view classification (Xu, Tao, and Xu 2013) et.al.”
• Evolutionary Generative Adversarial Networks (2018, neutral): “as video generation [11], [33], visual tracking [34]–[36], domain adaption [37], hashing coding [38]–[40], and feature learning [41], [42].”
• Triplet-Based Deep Hashing Network for Cross-Modal Retrieval (2018, methodology): “The MLP consists of three fullyconnected layers whose last layer is replaced with a new fch hash layer with k hidden nodes, in the same way as for the image modality in [25].”
• A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “In another work, the pairwise constraints are incorporated to learn the deep hashing network for cross-modal retrieval [144].”
• What Makes Multimodal Learning Better than Single (Provably) (2021, background): “Following the complementary principle [32], information from each single view can complement each other to improve the generalization ability of the whole learner on different tasks [14, 2, 34, 35].”
• Deep Bayesian Hashing with Center Prior for Multi-modal Neuroimage Retrieval (2020, methodology): “Pairwise relationship guided deep hashing (PGDH) [38] considers both intra-modal and inter-modal similarities and designs a regularization term to maximize the representation ability of the hash codes.”
• Two-Stream Deep Hashing With Class-Specific Centers for Supervised Image Search (2020, background): “Representative ANN-search solutions can be divided into tree-based [29], [30] and hash-based methods [17], [31]–[35].”
• Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval (2020, methodology): “For the supervised meth-ods [2, 3, 9, 15, 18, 19, 40, 42, 45], manually-annotated labels, requiring expensive human cost and computation, are leveraged to construct the similarity matrices and supervise hash code generation.”