Deep Supervised Cross-Modal Retrieval

By Liangli Zhen, Peng Hu, Xu Wang, Dezhong Peng. Computer Vision and Pattern Recognition, 2019

440 citations recorded by the source corpus.

Cross-modal retrieval aims to enable flexible retrieval across different modalities. The core of cross-modal retrieval is how to measure the content similarity between different types of data. In this paper, we present a novel cross-modal retrieval method, called Deep Supervised Cross-modal Retrieval (DSCMR). It aims to find a common representation space, in which the samples from different modalities can be compared directly. Specifically, DSCMR minimises the discrimination loss in both the label space and the common representation space to supervise the model learning discriminative features. Furthermore, it simultaneously minimises the modality invariance loss and uses a weight sharing strategy to eliminate the cross-modal discrepancy of multimedia data in the common representation space to learn modality-invariant features. Comprehensive experimental results on four widely-used benchmark datasets demonstrate that the proposed method is effective in cross-modal learning and significantly outperforms the state-of-the-art cross-modal retrieval methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b101e9d0e5a3b22335e9d2c4228c215fa4fcc8e4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Peng Hu
2. Xiaoqiang Yan
3. Jiawei Zhan
4. Liangli Zhen
5. Dezhong Peng
6. Shengsheng Qian
7. Long Ying
8. Pei Liu

Citations in the corpus

• Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval (2020, background): “Cross-modalsimilarityretrievalhasbeena popularresearch topic[15, 18, 20–22, 30, 31, 34, 34, 38, 39, 43, 46] with the objective to search the semantic similar instances from different modalities, which has already attracted attention in information retrieval [1, 23] and multimedia analysis [25, 26].”
• Scalable Deep Multimodal Learning for Cross-Modal Retrieval (2019, background): “Different from the existing cross-modal works [9, 10, 12, 38, 40, 44, 45], these modality-specific networks are able to be trained in parallel, and we only need to train a new modality-specific network to handle samples from a new modality.”
• Deep multi-view learning methods: A review (2021, methodology): “For the deep cross-modal retrieval methods, we compare the mean average precision (mAP) [162] of the following approaches: deep joint-semantics reconstructing hashing (DJSRH) [90], self-supervised 655 adversarial hashing (SSAH) [160], unsupervised deep cross-modal hashing (UDCMH) [159], unsupervised”