Supervised Coupled Dictionary Learning with Group Structures for Multi-modal Retrieval

By Yueting Zhuang, Yanfei Wang, Fei Wu, Yin Zhang, Weiming Lu. 2013

180 citations recorded by the source corpus.

A better similarity mapping function across heterogeneous high-dimensional features is very desirable for many applications involving multi-modal data. In this paper, we introduce coupled dictionary learning (DL) into supervised sparse coding for multi-modal (cross-media) retrieval. We call this Supervised coupled dictionary learning with group structures for Multi-Modal retrieval (SliM2). SliM2 formulates the multi-modal mapping as a constrained dictionary learning problem. By utilizing the intrinsic power of DL to deal with the heterogeneous features, SliM2 extends unimodal DL to multi-modal DL. Moreover, the label information is employed in SliM2 to discover the shared structure inside intra-modality within the same class by a mixed norm (i.e., `l1/l2`-norm). As a result, the multimodal retrieval is conducted via a set of jointly learned mapping functions across multi-modal data. The experimental results show the effectiveness of our proposed model when applied to cross-media retrieval.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer (2017, background): “…coupled dictionary learning has led to high performance algorithms for image super-resolution (Yang et al., 2010), allowing the reconstruction of high-res images from low-res samples, and for multi-modal retrieval (Zhuang, Wang, Wu, Zhang, & Lu, 2013) and cross-domain retrieval (Yu et al., 2014).”
A Survey of Multi-View Representation Learning (2016, unknown): “web data mining [122], as well as cross-media retrieval [123, 124].”
Learning Transferable Knowledge Through Embedding Spaces (2019, background): “This notion of coupled dictionary learning has led to high-performance algorithms for image super-resolution [11], allowing the reconstruction of high-res images from low-res samples, and for multi-modal retrieval [193], and cross-domain retrieval [65].”
Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment (2015, methodology): “One type of methods embeds images and text respectively into a common space where the correlation between a pair of highly relevant multi-modal data is minimized, such as canonical correlation analysis (CCA) [9] and multi-modal dictionary learning [28, 20, 31].”
Learning Cross-Modal Common Representations by Private–Shared Subspaces Separation (2020, methodology): “In addition to the CCA and its extensions, some other methods that utilize advanced schemes, such as factor analysis [25], dictionary learning [26], and half-quadratic optimization [27], to learn more effective common representations.”
Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval (2017, background): “In order to eliminate the diversity between different modality features, recent studies are concentrated on latent subspace learning [2], [3], [8], [10], [11] tries to learn a common latent subspace, where the heterogeneity between different modalities is minimized so that the learned features in th”
RGB-Infrared Cross-Modality Person Re-identification (2017, background): “one includes subspace learning methods [25, 30, 60] and deep learning frameworks [45, 6, 14, 12], while the latter one includes linear models [39, 36, 59, 51] and non-linear models [27, 50, 31].”
Adversarial Cross-Modal Retrieval (2017, methodology): “In this category, several subclasses of approaches can be distinguished: unsupervised [1, 5, 9, 29, 36], pair-wise [5, 11, 25, 43], ranking-based [33, 39] and supervised [7, 30, 31, 40] ones.”
Cross-modal Retrieval with Correspondence Autoencoder (2014, methodology): “Following [30], retrieval performance is evaluated using two metrics, mean average precision ( mAP ) and top 20% percentage.”
Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval (2016, methodology): “Besides these three kinds of methods, there are some other methods [40], [41], [42], [43], [44], [45] proposed for the cross-modal problem, such as the dictionary learning method [40], the graph-based learning method [42], the”
Partial Multi-Modal Sparse Coding via Adaptive Similarity Structure Regularization (2016, background): “Multimodal sparse coding is an essential problem for the representation of data with multiple modalities, which arises in many real applications such as cross-modal retrieval [16] and object recognition [4].”
Quantifying the Visual Concreteness of Words and Topics in Multimodal Datasets (2018, background): “(Khan et al., 2009; Socher and Fei-Fei, 2010; Jia et al., 2011; Zhuang et al., 2013, inter alia).”
Cross-Modal Similarity Learning: A Low Rank Bilinear Formulation (2014, background): “…the deep neural network (Ngiam et al. 2011; Srivastava and Salakhutdinov 2012), domain transfer (Duan, Xu, and Tsang 2012; Kulis, Saenko, and Dar-rell 2011; Socher et al. 2013), etc. (Bronstein et al. 2010; Jia, Salzmann, and Darrell 2011; Zhuang and Hoi 2011; Zhuang et al. 2013; Gong et al. 2014).”
Discriminative coupled dictionary hashing for fast cross-media retrieval (2014, background): “Cross-media retrieval is a hot research focus in recent years [16, 32, 31].”
Hierarchical Discriminative Learning for Visible Thermal Person Re-Identification (2018, methodology): “For learning matching models, dictionary learning1 (Peng et al. 2017; Zhuang et al. 2013; Das, Mandal, and Biswas 2017) and metric learning (Huo et al. 2017; Wang et al. 2016a; 2017) are two widely used techniques.”
A Comprehensive Survey on Cross-modal Retrieval (2016, methodology): “Inspired by the idea of (semi-)coupled dictionary learning, Zhuang et al. [49] bring coupled dictionary learning into supervised sparse coding for cross-modal retrieval, which is called Supervised coupled dictionary learning with group structures for multi-modal retrieval (SliM 2 ).”
An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges (2017, background): “[64] propose to extend single-media coupled dictionary learning to cross-media retrieval, assuming that there exist linear mappings among sparse coefficients of different media types.”