Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization

By Dongqing Zhang, Wu-Jun Li. AAAI Conference on Artificial Intelligence, 2014

640 citations recorded by the source corpus.

Due to its low storage cost and fast query speed, hashing has been widely adopted for similarity search in multimedia data. In particular, more and more attentions have been payed to multimodal hashing for search in multimedia data with multiple modalities, such as images with tags. Typically, supervised information of semantic labels is also available for the data points in many real applications. Hence, many supervised multimodal hashing~(SMH) methods have been proposed to utilize such semantic labels to further improve the search accuracy. However, the training time complexity of most existing SMH methods is too high, which makes them unscalable to large-scale datasets. In this paper, a novel SMH method, called semantic correlation maximization~(SCM), is proposed to seamlessly integrate semantic labels into the hashing learning procedure for large-scale data modeling. Experimental results on two real-world datasets show that SCM can significantly outperform the state-of-the-art SMH methods, in terms of both accuracy and scalability.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a09a072750bbf183412a62e60ba9c471e3ad2c70

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Yi Yang
3. Ling Shao
4. Luc Van Gool
5. Qi Tian
6. Xiaogang Wang
7. Jiashi Feng
8. Alan L. Yuille

Citations in the corpus

• Triplet-Based Deep Hashing Network for Cross-Modal Retrieval (2018, methodology): “Baselines We compare our proposed TDH method with eight stateof-the-art cross-modal hashing methods, namely CMFH [20], SCM [33], LSSH [21], STMH [39], CVH [32], SePH [23], DCMH [24] and PRDH [25], where DCMH and PRDH are deep methods and the rest are shallow methods.”
• Self-Supervised Learning from Web Data for Multimodal Retrieval (2019, methodology): “Method GloVe tf-idf MMSHL [44] SCM [39] GloVe tf-idf Train MIRFlickr InstaCities animals 0.”
• Multimodal Machine Learning: A Survey and Taxonomy (2017, methodology): “Coordinated Similarity Images + Text [61], [105] Video + Text [159], [231] Structured Images + Text [32], [212], [248] Audio + Articulatory [220]”
• Deep Bayesian Hashing with Center Prior for Multi-modal Neuroimage Retrieval (2020, methodology): “…cross-modal hashing methods based on traditional machine learning techniques, including ACQ [34], STMH [33], CVH [25], SEPH [27], and SCM [36] and then compared with three recently proposed deep learning methods for cross-modal hashing, including PGDH [38], DCMH [37], and CMHH [26].”
• SCH-GAN: Semi-Supervised Cross-Modal Hashing by Generative Adversarial Network (2018, methodology): “6) SCM [16] constructs semantic similarity matrix based on labels and learns hashing functions to preserve the constructed matrix.”
• A Survey of Multi-View Representation Learning (2016, neutral): “To make Bayesian CCA feasible for high-dimensional data, Huopaniemi et al. [71] propose to use multi-way analysis setup by exploiting dimensionality reduction techniques.”
• Cross-Modal Retrieval With CNN Visual Features: A New Baseline (2017, background): “To address the problem of prohibitively expensive nearest neighbor search, some hashingbased approaches [3], [26], [28], [43], [44], [59], [65], [67], [69], [70] to large scale similarity search have drawn much interest from the cross-modal retrieval community.”
• Zero-Shot Sketch-Image Hashing (2018, methodology): “As an extension of conventional data hashing techniques [51, 20, 39, 54, 53], cross-modal hashing [4, 13, 72, 36, 33, 26, 5, 6] show great potential in retrieving heterogeneous data with high efficiency due to the computationally cheap Hamming space matching, which is recently adopted to large-scale”
• RGB-Infrared Cross-Modality Person Re-identification (2017, methodology): “Metric learning methods included KISSME [15], LFDA [34] and cross-domain models CCA [35], CDFE [25], GMA [37], SCM [51] and CRAFT [3].”
• Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval (2017, methodology): “Zhang and Li [28] proposed semantic correlation maximization (SCM) to seamlessly integrate semantic labels into the hashing learning procedure for large-scale data modeling.”
• Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval (2018, methodology): “Supervised hashing methods [2, 4, 14, 16, 39, 40, 44] aim to exploit available supervised information (such as labels or the semantic affinities of training data) to improve performance.”
• Deep Visual-Semantic Hashing for Cross-Modal Retrieval (2016, methodology): “This work is related to cross-modal hashing, which has been an increasingly popular research topic in machine learning, data mining, and multimedia retrieval communities [4, 22, 44, 45, 33, 31, 30, 37, 38, 41, 8, 16, 43, 27, 39, 25].”
• Shared Predictive Cross-Modal Deep Quantization (2018, neutral): “umbers of retrieved instances. Compared methods. We compare the proposed approach with various state-of-the-art cross-modal similarity search methods. Specifically, we take six supervised methods: SCM [73], SePH [42], CMSSH [27], CVH [33], DCMH [21], CDQ [52], as baselines, where DCMH is deep hashing”
• Deep Cross-Modal Hashing (2016, methodology): “…latent binary embedding (MLBE) (Zhen & Yeung, 2012a), co-regularized hashing (CRH) (Zhen & Yeung, 2012b), semantic correlation maximization (SCM) (Zhang & Li, 2014), collective matrix factorization hashing (CMFH) (Ding et al., 2014), semantic topic multi-modal hashing (STMH) (Wang et al., 2015)…”
• Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing (2020, methodology): “4 of them used additional supervision (CMSSH [1], SCM [44], DCMH [17], and SSAH [20]), and while 5 others (CVH [19], PDH [28], CMFH [9], and CCQ [26]), and UGACH [45]) did not.”
• Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval (2020, methodology): “For the supervised methods [2, 3, 9, 15, 18, 19, 40, 42, 45], manually-annotated labels, requiring expensive human cost and computation, are leveraged to Session 8B: Multi-modal Retrieval and Ranking SIGIR ’20, July 25–30, 2020, Virtual Event, China”
• Semantics-preserving hashing for cross-view retrieval (2015, methodology): “Unlike previous work [9, 26, 19] that utilizes the supervised information for independently weighting each pairwise distance/similarity between hash codes, SePH standardizes all Hamming distances by transforming each into a probability that depends on all others.”