A Survey of Multi-View Representation Learning

By Yingming Li, Ming Yang, Zhongfei Zhang. 2016

640 citations recorded by the source corpus.

Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then, from the perspective of representation fusion, we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yonglong Tian
2. Dilip Krishnan
3. Phillip Isola
4. Jing Gao
5. Peng Hu
6. Xi Peng
7. Peng Li
8. Wenzhong Guo

In Computer Vision and Pattern Recognition

1. Yonglong Tian
2. Luc Van Gool
3. Ming-Hsuan Yang
4. Xi Li
5. Dacheng Tao
6. Phillip Isola
7. Yi Yang
8. Xiaogang Wang

In Representation & Self-Supervised Learning

1. Phillip Isola
2. Yonglong Tian
3. Dilip Krishnan
4. Jing Gao
5. Peng Hu
6. Xi Peng
7. Peng Li
8. Wenzhong Guo

Citations in the corpus

• Contrastive Multiview Coding (2019, background): “a of multi-view learning, which is a general term involving many different approaches such as co-training [6], multi-kernel learning [11] and metric learning [4]; for comprehensive surveys please see [60, 34]. Nearly all existing works have dealt with one or two views such as video or image/sound. H”
• Multimodal and Temporal Perception of Audio-visual Cues for Emotion Recognition (2019, background): “graphical models [21] and neural network models”
• Robust Multi-View Clustering With Incomplete Information (2022, background): “MULTI-VIEW clustering (MvC) [1], [2], [3], [4] aims at learning a common representation for multi-view data and then employing clustering on the representation.”
• Recent Advances and Trends in Multimodal Deep Learning: A Review (2021, background): “1 Multimodal Intelligence: Representation Learning, Information Fusion, and Applications [174] IEEE 2020 2 Experience Grounds Language [16] arXiv 2020 3 A Survey on Deep Learning for Multimodal Data Fusion [40] NC(MIT) 2020 4 Trends in integration of vision and language research: A survey of tasks, ”
• Deep Multimodal Representation Learning: A Survey (2019, background): “[19] which concentrates on multi-view representation learning, including shallow and deep methods, while, by contrast, ours highlight the latter which have gained more attention in recent years.”
• A Survey on Deep Learning for Multimodal Data Fusion (2020, background): “And each modality of different source, type, and distribution contains modality-specific information (Li, Yang, & Zhang, 2019; Gao, Li, & Li, 2016).”
• Deep multi-view learning methods: A review (2021, background): “These surveys focus on the problems of specific MVL methods, such as multi-view fusion [10, 27, 28], multi-modal learning [37, 38], multi-view clustering [26] and multi-view representation learning [39, 40].”