High-Dimensional Sparse Cross-Modal Hashing with Fine-Grained Similarity Embedding

By Yongxin Wang, Zhen-Duo Chen, Xin Luo, Xin-Shun Xu. 2021

22 citations recorded by the source corpus.

Recently, with the discoveries in neurobiology, high-dimensional sparse hashing has attracted increasing attention. In contrast with general hashing that generates low-dimensional hash codes, the high-dimensional sparse hashing maps inputs into a higher dimensional space and generates sparse hash codes, achieving superior performance. However, the sparse hashing has not been fully studied in hashing literature yet. For example, how to fully explore the power of sparse coding in cross-modal retrieval tasks; how to discretely solve the binary and sparse constraints so as to avoid the quantization error problem. Motivated by these issues, in this paper, we present an efficient sparse hashing method, i.e., High-dimensional Sparse Cross-modal Hashing, HSCH for short. It not only takes the high-level semantic similarity of data into consideration, but also properly exploits the low-level feature similarity. In specific, we theoretically design a fine-grained similarity with two critical fusion rules. Then we take advantage of sparse codes to embed the fine-grained similarity into the to-be-learnt hash codes. Moreover, an efficient discrete optimization algorithm is proposed to solve the binary and sparse constraints, reducing the quantization error. In light of this, it becomes much more trainable, and the learnt hash codes are more discriminative. More importantly, the retrieval complexity of HSCH is as efficient as general hash methods. Extensive experiments on three widely-used datasets demonstrate the superior performance of HSCH compared with several state-of-the-art cross-modal hashing approaches.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Chenliang Li
2. Songfang Huang
3. Bin Bi
4. Ming Yan
5. Fei Huang
6. Wei Wang
7. Haiyang Xu
8. Luo Si

In Multimodal Learning

1. Fei Huang
2. Chenliang Li
3. Haiyang Xu
4. Ming Yan
5. Ji Zhang
6. Weihua Chen
7. Fan Wang
8. Songfang Huang

In Representation & Self-Supervised Learning

1. Fei Huang
2. Chenliang Li
3. Haiyang Xu
4. Ming Yan
5. Luo Si
6. Ji Zhang
7. Weihua Chen
8. Fan Wang

Citations in the corpus

• Achieving Human Parity on Visual Question Answering (2021, background): “Visual question answering involves the cross-modal fusion and alignment between image and text, which is related to the topic of cross-modal retrieval [12, 26, 28, 32, 47, 60, 61, 84, 92].”