Statistical Consistency of Kernel Canonical Correlation Analysis

By Francis R. Bach, Francis Bach@mines Org, Dé. 2007

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “See, e.g., Fukumizu et al. (2007; 2008).”
The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA (2019, background): “Nonlinear extensions of the basic CCA framework have been proposed [26, 13, 3, 29], but identifiability results in the sense we consider in this paper are lacking.”
A low variance consistent test of relative dependency (2014, methodology): “M L ] 1 variables and the source is computed using the Hilbert-Schmidt Independence Criterion [4, 5].”
Kernel Mean Shrinkage Estimators (2014, methodology): “…instance, nonlinear component analyses, such as kernel principal component analysis (KPCA), kernel Fisher discriminant analysis (KFDA), and kernel canonical correlation analysis (KCCA), rely heavily on mean functions and covariance operators in RKHS (Sch¨olkopf et al., 1998; Fukumizu et al., 2007).”
Density Estimation in Infinite Dimensional Exponential Families (2013, neutral): “It follows from ‖(Ĉ + λI)−1‖ ≤ 1/λ and ‖Ĉ − C‖ ≤ ‖Ĉ − C‖HS = Op0(n) (where the ideas in the proof of Lemma 5 in [15] can be used to prove this claim under the assumption that the first term in (D) holds with ε = 2) that the first term is of the order Op0 ( A0(λ) λ √ n ) .”
A Kernel Test for Three-Variable Interactions (2013, background): “Much of the work in this area focuses on measuring or testing pairwise interaction: for instance, the Hilbert-Schmidt Independence Criterion (HSIC) or Distance Covariance [1, 2, 3], kernel canonical correlation [4, 5, 6], and mutual information [7].”
Testing Hypotheses by Regularized Maximum Mean Discrepancy (2013, background): “Given samples {xi} i=1 and {yi} n2 i=1 drawn from distributions P and Q, respectively, the mean element, the cross-covariance operator and the covariance operator are defined as follows [7], [9]: μ̂P = 1 n1 ∑n1 i=1 k(xi, .”
Kernel dimension reduction in regression (2009, methodology): “, F,F 1,...,F n and G,G 1,...,G n are zero mean i.i.d. random elements in HX and HY, respectively. In the following, the notation F =HX ⊗HY is used for simplicity. As shown in the proof of Lemma 4 in [13], we have kΣb(n) Y X −Σ Y Xk HS = 1 n Xn i=1 F i − 1 n Xn j=1 F j ! G i − 1 n Xn j=1 G j ! −E[FG”
A Survey on Multi-view Learning (2013, methodology): “Fukumizu et al. (2007) investigated the general problem of establishing a consistency of KCCA by providing the rates for the regularization parameter, and proved that when”
Data-Driven Intelligent Transportation Systems: A Survey (2011, background): “For example, both canonical correlation [116] and Procrustes analyses [117] are devoted to aligning two heterogeneous data into a common space.”
Kernel-Based Methods for Hypothesis Testing: A Unified View (2013, unknown): “The largest singular value is the largest kernel canonical correlation [36], [58], while its Hilbert-Schmidt norm leads to comparing distributions with D p p p , x y x y 2 | ^ h [59].”
Nonparametric Canonical Correlation Analysis (2015, background): “Namely, KCCA has been shown to converge to the optimal RKHS projections when the kernel bandwidths are fixed (Fukumizu et al., 2007) (but not to the population solution (7) for bandwidths tending to 0).”
Learning structured densities via infinite dimensional exponential families (2015, unknown): “1 (Lemma 5 in [8] or Theorem 5 in [26] ) (Properties of C) 1.”
Nonparametric Tree Graphical Models via Kernel Embeddings (2010, methodology): “Note that we have added an additional regularizing term λ, to avoid overfitting (in much the same way as is done for kernel canonical correlation analysis: see Fukumizu et al., 2007).”
Kernel Change-point Analysis (2008, methodology): “This is described in the operator-theoretic framework, developed for the statistical analysis of kernel-based lea rning and testing algorithms in [10, 11 ].”
Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment (2013, background): “While an appropriate choice of the regularization parameter may overcome the latter issue (Fukumizu et al., 2007), the tuning parameter selection problem for KCCA has not been adequately addressed in the literature.”