On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval

By José Costa Pereira, Emanuele Coviello, Gabriel Doyle, Nikhil Rasiwasia, Gert R. G. Lanckriet, Roger Levy, Nuno Vasconcelos. 2014

399 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Information Retrieval & Search, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Yi Yang
3. Luc Van Gool
4. Ling Shao
5. Ming-Hsuan Yang
6. Shuicheng Yan
7. Xiaogang Wang
8. Jiashi Feng

In Information Retrieval & Search

1. Yuxin Peng
2. Liang Wang
3. Shuicheng Yan
4. Dacheng Tao
5. Tao Mei
6. Ling Shao
7. Xiaojun Chang
8. Bo Du

In Multimodal Learning

1. Yuxin Peng
2. Heng Tao Shen
3. Xing Xu
4. Dacheng Tao
5. Yang Yang
6. Alan Hanjalic
7. Bokun Wang
8. Liang Wang

Citations in the corpus

• Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces (2017, neutral): “Methods such as those presented in [29, 13, 27, 22] use Canonical Correlation Analysis (CCA) for establishing relationships between data of different modalities.”
• Image Captioning with Deep Bidirectional LSTMs (2016, background): “This is an instance of cross-modal retrieval [6, 30, 41] which has been a hot research subject in multimedia field.”
• Adversarial Cross-Modal Retrieval (2017, methodology): “We conduct experiments on four widely-used cross-modal datasets: Wikipedia dataset [4], NUS-WIDE-10k dataset [3], Pascal Sentence dataset [23], and MSCOCO dataset [16].”
• Deep Cross-Modal Audio-Visual Generation (2017, background): “Our work differs from other various work in cross-modal retrieval [6, 19, 21, 32] as stated in Sec.”
• Deep Supervised Cross-Modal Retrieval (2019, methodology): “To verify the effectiveness of the proposed method, we conduct experiments on four widely-used benchmark datasets: the Wikipedia dataset [24], the Pascal Sentence dataset [26], the NUS-WIDE-10k dataset [4] and the XMediaNet dataset [20, 23].”
• Deep Collaborative Embedding for Social Image Understanding (2019, methodology): “In [9], three strategies are designed for the cross-modal retrieval problem: the unsupervised lowlevel cross-modal correlations, supervised semantic matching and semantic correlation matching combining the above two”
• Quantifying the Visual Concreteness of Words and Topics in Multimodal Datasets (2018, methodology): “Applying machine learning to understand visual-textual relationships has enabled a number of new applications, e.g., better accessibility via automatic generation of alt text (Garcia et al., 2016), cheaper training-data acquisition for computer vision (Joulin et al., 2016; Veit et al., 2017), and cr”
• A Survey of Multi-View Representation Learning (2016, neutral): “Several extensions of CFA are presented by incorporating non-linearity and supervised information [111], [112], [113].”
• Cross-Modality Submodular Dictionary Learning for Information Retrieval (2014, methodology): “…the non-knowledge transfer method (PCA), correlation matching (CM) [ 31 ] , semantic matching (SM) [ 31 ] , semantic correlation matching (SCM) [ 7 ] , a bilinear model (BLM) [ 42 ] , generalized multiview linear discriminant analysis (GMLDA) [ 36 ]…”
• Teaching Semi-Supervised Classifier via Generalized Distillation (2018, methodology): “To this end, we use a recent Wikipedia dataset [ Pereira et al. , 2014 ] that contains 2866 documents across 10 categories such as “history”, “music”, and “biology”.”
• A Comprehensive Survey on Cross-modal Retrieval (2016, methodology): “It has been widely used for cross-media retrieval[13], [84], [85], cross-lingual retrieval [86] and some vision problems [87].”
• Cross-Modal Similarity Learning: A Low Rank Bilinear Formulation (2014, methodology): “In the work of (Rasiwasia et al. 2010; Costa Pereira et al. 2014), Rasiwasia et al. proposed a semantic correlation matching (SCM) algorithm to deal with the problem.”
• Learning Consistent Feature Representation for Cross-Modal Multimedia Retrieval (2015, methodology): “In this section, the proposed algorithm is applied to the cross-modal retrieval problem on four databases, the Pascal VOC2007 [11], the Wikipedia [30], the LabelMe [27], and the NUS-WIDE databases [2], which have been widely used in the cross-media retrieval field [3], [10], [22], [33], [42], [44].”
• Semantics-preserving hashing for cross-view retrieval (2015, neutral): “Wiki [12], MIRFlickr [7] and NUS-WIDE [2].”
• An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges (2017, background): “With the great advance of deep learning, deep neural network (DNN) has shown its potential in different multimedia applications such as object recognition [31] and text generation [32].”
• Cross-Modal Retrieval With CNN Visual Features: A New Baseline (2017, background): “Recently, based on CCA, many extensions [5], [13], [38], [39], [42], [56] are applied to crossmodal retrieval.”
• SCH-GAN: Semi-Supervised Cross-Modal Hashing by Generative Adversarial Network (2018, methodology): “eral baseline experiments to investigate the performance of generative and discriminative model. A. Dataset We evaluate the proposed approach and compared methods on 3 widely-used datasets: Wikepedia [42], NUSWIDE [43] and MIRFLICKR [44]. We’ll briefly introduce three datasets. Wikipedia dataset [42”
• Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding (2015, methodology): “In this paper, we select three benchmark data sets for performance evaluation of our EMR-SLRAalgorithm: the Wiki Text-Image data set(1) [34], the NUS-WIDE-LITE data set(2) [35], and the HYDICE hyperspectral data set(3) [36].”