Nonparametric Canonical Correlation Analysis

By Tomer Michaeli, Weiran Wang, Karen Livescu. 2015

92 citations recorded by the source corpus.

Canonical correlation analysis (CCA) is a classical representation learning technique for finding correlated variables in multi-view data. Several nonlinear extensions of the original linear CCA have been proposed, including kernel and deep neural network methods. These approaches seek maximally correlated projections among families of functions, which the user specifies (by choosing a kernel or neural network structure), and are computationally demanding. Interestingly, the theory of nonlinear CCA, without functional restrictions, had been studied in the population setting by Lancaster already in the 1950s, but these results have not inspired practical algorithms. We revisit Lancaster's theory to devise a practical algorithm for nonparametric CCA (NCCA). Specifically, we show that the solution can be expressed in terms of the singular value decomposition of a certain operator associated with the joint density of the views. Thus, by estimating the population density from data, NCCA reduces to solving an eigenvalue system, superficially like kernel CCA but, importantly, without requiring the inversion of any kernel matrix. We also derive a partially linear CCA (PLCCA) variant in which one of the views undergoes a linear projection while the other is nonparametric. Using a kernel density estimate based on a small number of nearest neighbors, our NCCA and PLCCA algorithms are memory-efficient, often run much faster, and perform better than kernel CCA and comparable to deep CCA.

Subfields: Artificial Intelligence, Computer Networks and Communications, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Hung-yi Lee
3. Dong Yu
4. Shinji Watanabe
5. Jinyu Li
6. James R. Glass
7. Tara N. Sainath
8. Abdel-rahman Mohamed

In Computer Networks and Communications

1. Lior Wolf
2. Aviv Eisenschtat
3. Hung-yi Lee
4. Bernhard Schölkopf
5. Abdel-rahman Mohamed
6. Francesco Locatello

In Computer Vision and Pattern Recognition

1. Bernhard Schölkopf
2. Luc Van Gool
3. Dacheng Tao
4. Lior Wolf
5. Trevor Darrell
6. Xiaogang Wang
7. Ming-Hsuan Yang
8. Yi Yang

Citations in the corpus

• The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, background): “Nonlinear extensions of the basic CCA framework have been proposed [26, 13, 3, 29], but identifiability results in the sense we consider in this paper are lacking.”
• Linking Image and Text with 2-Way Nets (2016, methodology): “There are a large number of CCA variants including: regularized CCA [45], Nonparametric canonical correlation analysis (NCCA) [32], and Kernel canonical correlation analysis (KCCA) [2, 31, 5], a method for producing non-linear, non-parametric projections using the kernel trick.”
• Self-Supervised Speech Representation Learning: A Review (2022, methodology): “Some typical approaches in this category include Multi-view autoencoders and variations [154], [155], Multi-modal deep Boltzmann machines [156], Canonical correlation analysis (CCA) [157] and its nonlinear extensions [158], [159], [160], [161], [162], [163], [164], [165], [166], Multi-view contrasti”