Deep Canonical Correlation Analysis

By Galen Andrew, Raman Arora, J. Bilmes, Karen Livescu. 2013

2,085 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

A Retrieve-and-Edit Framework for Predicting Structured Outputs (2018, background): “A related idea is cross-modal embeddings, which jointly embed and align items in different domains (such as images and captions) using autoencoders [39, 2, 31, 10].”
Grounded Compositional Semantics for Finding and Describing Images with Sentences (2014, methodology): “Technically, one could combine the recently introduced deep CCA Andrew et al. (2013) and train the recursive neural network architectures with the CCA objective.”
On Mutual Information Maximization for Representation Learning (2019, methodology): “This setup has been used in deep canonical correlation analysis papers (Andrew et al., 2013), where the target is to maximize the correlation between the representations.”
Achieving Human Parity on Visual Question Answering (2021, background): “For example, Sharma et al. [66] extend CCA to a supervised version by leveraging the the semantic category labels to guide the representation learning.”
Learning Deep Structure-Preserving Image-Text Embeddings (2015, methodology): “From the viewpoint of architecture, our method is similar to the two-branch Deep CCA models [1, 33], though it avoids Deep CCA’s training-time difficulties associated with covariance matrix estimation.”
Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning (2017, neutral): “In particular, Arandjelovi´c and Zisserman (2017) solved a similar unsupervised learning problem, but instead of using hand-crafted audio features, they jointly learned audio and visual CNNs.”
The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, background): “Nonlinear extensions of the basic CCA framework have been proposed [26, 13, 3, 29], but identifiability results in the sense we consider in this paper are lacking.”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, background): “[1] proposed a deep canonical correlation analysis network with two stacks of deep embedding followed by a CCA on top layer.”
Learning Two-Branch Neural Networks for Image-Text Matching Tasks (2017, background): “[25] and Yan and Mikolajczyk [26] proposed to cast CCA into a deep learning framework.”
Ambient Sound Provides Supervision for Visual Learning (2016, neutral): “Also, in contrast to other recent work in multi-modal representation learning [28,33,2], our technical approach is based on solving a self-supervised classification problem (rather than a generative model or autoencoder), and our goal is to learn visual representations that are generally useful for ”
Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks (2015, background): “A variety of other neural network models have been proposed for similarity tasks (Weston et al., 2011; Huang et al., 2013; Andrew et al., 2013; Bromley et al., 1993).”
Attention-based Multimodal Neural Machine Translation (2016, background): “Rather than using the embedding of each modality independently, Some works (Hardoon et al., 2004; Andrew et al., 2013; Ngiam et al., 2011; Srivastava and Salakhutdinov, 2014) focus on learning joint space of different modalities.”
Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement (2016, background): “There are many other examples of neural network-based sentence modeling approaches for the STS problem (Yin and Sch¨utze, 2015; Huang et al., 2013; Andrew et al., 2013; Weston et al., 2011; Socher et al., 2011; Zarrella et al., 2015).”
Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition (2021, methodology): “In the original DCCA [34], the authors did not explicitly describe how to use transformed features for real-world applications via machine learning algorithms.”
Deep Visual-Semantic Hashing for Cross-Modal Retrieval (2016, background): “Deep architectures are very powerful for extracting the multimodal embedding shared by diﬀerent modalities since they can extract nonlinear feature representations to bridge diﬀerent modalities eﬀectively [2, 9, 34, 19, 20, 6, 18].”
Coincidence, Categorization, and Consolidation: Learning to Recognize Sounds with Minimal Supervision (2019, methodology): “Three prominent deep learning approaches are (i) Deep Canonical Correlation Analysis (DeepCCA) [26], a deep learning generalization of linear CCA; (ii) Look, Listen, and Learn ( L 3 ) [12], which learns representations that can predict audio-visual frame correspondence in videos; and (iii) metric…”
A Comprehensive Survey on Pose-Invariant Face Recognition (2015, methodology): “These include the lasso regularization Annan et al. (2012); Zhang et al. (2013b), the ridge regularization Annan et al. (2012), local similarity regularization Hao and Qi (2015), and neighborhood consistency regularization Hao and Qi (2015). Based on a similar assumption to the above works, Yin et a”
Training Deep Networks with Structured Layers by Matrix Backpropagation (2015, background): “Some examples are in the context of camera calibration [32], as mentioned, for learning parameters in a normalized cuts model[13], learning the parameters of Gaussian CRFs for denoising [33] and learning deep canonical correlation models [34].”
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting (2020, positive): “For learning VSE, we leverage the contrastive loss (Kiros et al., 2014) from cross-modal retrieval, which is shown more robust than maximizing canonical correlation among modalities as in (Andrew et al., 2013; Huang et al., 2018).”