Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval

By Chao Li, Cheng Deng, Ning Li, Wei David Liu, Xinbo Gao, Dacheng Tao. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

410 citations recorded by the source corpus.

Thanks to the success of deep learning, cross-modal retrieval has made significant progress recently. However, there still remains a crucial bottleneck: how to bridge the modality gap to further enhance the retrieval accuracy. In this paper, we propose a self-supervised adversarial hashing (SSAH) approach, which lies among the early attempts to incorporate adversarial learning into cross-modal hashing in a self-supervised fashion. The primary contribution of this work is that two adversarial networks are leveraged to maximize the semantic correlation and consistency of the representations between different modalities. In addition, we harness a self-supervised semantic network to discover high-level semantic information in the form of multi-label annotations. Such information guides the feature learning process and preserves the modality relationships in both the common semantic space and the Hamming space. Extensive experiments carried out on three benchmark datasets validate that the proposed SSAH surpasses the state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a6d0a248e37ad54ccc4277605118ba980266f23e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Qi Tian
3. Yi Yang
4. Cheng Deng
5. Luc Van Gool
6. Lingxi Xie
7. Alan L. Yuille
8. Ming-Hsuan Yang

Citations in the corpus

• Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer (2020, background): “Cross-modality retrieval has been widely studied for heterogeneous face recognition [15] and text-to-image retrieval [9, 16, 21, 22, 27].”
• Deep Spectral Clustering Using Dual Autoencoder Network (2019, methodology): “[50] presents a deep adversarial subspace clustering (DASC) method to learn more favorable representations and supervise sample representation learning by adversarial deep learning [21].”
• Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval (2020, methodology): “Hashing based cross-modal retrieval methods [15, 17, 18, 20–22, 30, 31, 38, 39, 47] largely improve the retrieval efficiency on both speed and storage by mapping large-scale and high-dimensional multi-modal media data into a common Hamming space, where semantic similar instances from different…”
• Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing (2020, methodology): “, using adversarial-learning-based approaches [20, 21, 13]), but they differ from each other in whether an instance-level annotation is provided during the training stage.”
• Deep Bayesian Hashing with Center Prior for Multi-modal Neuroimage Retrieval (2020, background): “Deep learning-based cross-modal hashing methods [26], [37]–[39] have also been proposed for image retrieval.”
• A Decade Survey of Content Based Image Retrieval Using Deep Learning (2020, methodology): “GAN has been also used for cross-modal retrieval [143], [145], [146], [160], semi-supervised hashing [147], [161], sketch based image retrieval [148], [159], [163] and unsupervised adversarial hashing [122].”
• High-Dimensional Sparse Cross-Modal Hashing with Fine-Grained Similarity Embedding (2021, background): “, DCMH [19], SSAH [24], EGDH [42], MLCAH [36], and DADH [1].”
• Unsupervised Semantic-Preserving Adversarial Hashing for Image Search (2019, methodology): “Representative ANN solutions include tree-based [18], [19] and hash-based methods [8], [20]–[28].”
• Deep multi-view learning methods: A review (2021, methodology): “For the deep cross-modal retrieval methods, we compare the mean average precision (mAP) [162] of the following approaches: deep joint-semantics reconstructing hashing (DJSRH) [90], self-supervised 655 adversarial hashing (SSAH) [160], unsupervised deep cross-modal hashing (UDCMH) [159], unsupervised”