Deep Variational Canonical Correlation Analysis

By Weiran Wang, Honglak Lee, Karen Livescu. 2016

155 citations recorded by the source corpus.

We present deep variational canonical correlation analysis (VCCA), a deep multi-view learning model that extends the latent variable model interpretation of linear CCA to nonlinear observation models parameterized by deep neural networks. We derive variational lower bounds of the data likelihood by parameterizing the posterior probability of the latent variables from the view that is available at test time. We also propose a variant of VCCA called VCCA-private that can, in addition to the "common variables" underlying both views, extract the "private variables" within each view, and disentangles the shared and private information for multi-view data without hard supervision. Experimental results on real-world datasets show that our methods are competitive across domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Variational Information Bottleneck (2017, methodology): “The method of Wang et al. (2016b) proposes a latent variable generative model of both x and y ; their variational lower bound is closely related to ours, with the following differences.”
SCAN: Learning Hierarchical Compositional Visual Concepts (2017, background): “…& Dukkipati, 2017) ; and 3) multimodal generative models that aim to embed visual and symbolic inputs in a joint latent space in order to be able to run bi-directional inferences (Vedantam et al., 2017; Suzuki et al., 2017; Pu et al., 2016; Wang et al., 2016; Srivastava & Salakhutdinov, 2014).”
Trusted Multi-View Classification With Dynamic Evidential Fusion (2022, unknown): “CCA-based multiview models [36], [37], [38], [39], [40], [41] are representative”
Self-Supervised Speech Representation Learning: A Review (2022, methodology): “Some typical approaches in this category include Multi-view autoencoders and variations [154], [155], Multi-modal deep Boltzmann machines [156], Canonical correlation analysis (CCA) [157] and its nonlinear extensions [158], [159], [160], [161], [162], [163], [164], [165], [166], Multi-view contrasti”
The Human Tumor Atlas Network: Charting Tumor Transitions across Space and Time at Single-Cell Resolution (2020, unknown): “Progress on similar problems in other computational domains could provide guidance (Jha et al., 2018; Ma et al., 2019; Wang et al., 2016).”
Generative Models of Visually Grounded Imagination (2017, methodology): “(Wang et al., 2016; Pu et al., 2016; Suzuki et al., 2017) all use exactly the same generative model as us, namely p ( x, y, z ) = p ( z ) p ( x | z ) p ( y | z ) .”
Deep Partial Multi-View Learning (2020, methodology): “The representative multi-view representation learning methods are CCA based, including kernelized CCA [10], deep neural network based CCA [11], [26], DVCCA [27] and semi-paired and semi-supervised generalized correlation analysis (S 2 GCA) [28].”
Learning Robust Representations via Multi-View Information Bottleneck (2020, methodology): “Following standard procedure in the literature (Srivastava & Salakhutdinov, 2014; Wang et al., 2016), we train our model on the unlabeled pairs of images and tags.”
SCAN: Learning Abstract Hierarchical Compositional Visual Concepts (2017, background): “…Sohn et al., 2015; Pandey & Dukkipati, 2017) ; and 3) multimodal generative models that aim to embed visual and symbolic inputs in a joint latent space in order to be able to run bi-directional inferences (Srivastava & Salakhutdinov, 2014; Suzuki et al., 2017; Pu et al., 2016; Wang et al., 2016).”
Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018, unknown): “We compare existing models (VAE, BiVCCA, JMVAE) to the MVAE and show that we equal state-of-the-art performance.”
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019, methodology): “In recent years, there have been a few approaches involving both VAEs and GANs that enable cross-modal generation both ways (Wang et al., 2016; Zhu et al., 2017a,b), but ignore learning a common embedding between them, instead treating the different cross-generations as independent but composable…”
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis (2020, methodology): “[55] revisited this framework by proposing a probabilistic CCA – deep variational CCA.”
Variational Deep Semantic Hashing for Text Documents (2017, methodology): “Following the practice in [37], we use the dropout technique [30] with the keep probability of 0.8 in training to alleviate overﬁtting. number of hidden nodes of the models is 1,500 for RCV1 and 1,000 for the other three smaller datasets.”
Deep Multimodal Representation Learning: A Survey (2019, unknown): “[120] proposed an extension named deep variational canonical correlation analysis (VCCA).”