Nonlinear Component Analysis as a Kernel Eigenvalue Problem

By Bernhard Schölkopf, Alex Smola, K. Müller. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

A Generative Process for Contractive Auto-Encoders (2012, background): “…PCA corresponds to a Gaussian model, where the variances in directions orthogonal to the manifold are small and constant (across all these directions), and this can be extended to non-linear structures with a mixture model (Tip-ping and Bishop, 1999) or the kernel trick (Sch¨olkopf et al., 1998).”
Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval (2015, methodology): “, 1990), ESA (Gabrilovich and Markovitch, 2007), PCA (Karhunen, 1998), and non-linear kernel variants (Schölkopf et al., 1998).”
Neural Tangent Kernel: Convergence and Generalization in Neural Networks (2018, background): “, f (NnL) of the data with respect to to the kernel K [17, 18].”
Representation Learning: A Review and New Perspectives (2012, methodology): “this geometric perspective adopt a non-parametric approach, based on a training set nearest neighbor graph (Schölkopf et al., 1998; Roweis and Saul, 2000; Tenenbaum et al., 2000; Brand, 2003; Belkin and Niyogi, 2003; Donoho and Grimes, 2003; Weinberger and Saul, 2004; Hinton and Roweis, 2003; van de”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “Many classical learning algorithms—such as the perceptron (Rosenblatt 1958), support vector machine (SVM) (Cortes and Vapnik 1995), and principal component analysis (PCA) (Pearson 1901, Hotelling 1933)—employ data instances, e.g., x,x′ ∈ X , only through an inner product 〈x,x′〉, which basically is a”
Security analysis of online centroid anomaly detection (2010, methodology): “When the latter is smaller than the size of the training data, an attacker can compute a PCA of the data matrix (Schölkopf et al., 1998) and project the original data into the subspace spanned by a smaller number of informative components.”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, methodology): “The diﬀerence between PCA and FA is similar to that between the basic AE and the variational AE: the latter assumes that hypothetical, underlying variables exist and cause the observed data.”
Causal Discovery from Heterogeneous/Nonstationary Data (2019, methodology): “This can be achieved by exploiting KPCA techniques (Schölkopf et al., 1998), which computes principal components in kernel spaces of the input.”
Behind Distribution Shift: Mining Driving Forces of Changes and Causal Arrows (2017, neutral): “That is, perform eigenvalue decomposition on M to find the nonlinear principal components λ̂i(C) [8].”
Kernel Sequential Monte Carlo (2015, methodology): “ependencies across dimensions, adaptation of a global (location-independent) covariance matrix as in [9] can be ineﬃcient. For this reason, in a spirit similar to Kernel Principal Components Analysis [16], [2] represent covariance of the target in a (possibly inﬁnite-dimensional) RKHS, induced by a ”
Fastfood: Approximate Kernel Expansions in Loglinear Time (2014, methodology): “This trick that was also exploited by Schölkopf et al. (1998) for evaluating PCA.”
Kernel Mean Shrinkage Estimators (2014, methodology): “For f An RKHS over a set X is a Hilbert space H consisting of functions on X such that for each x ∈ X there is a function k x ∈ H with the property (5) The function k x ( ) k ( x, · ) is called the reproducing kernel of H and the equality (5) is called the reproducing property of H .”
Kernel Adaptive Metropolis-Hastings (2013, methodology): “Now, we can construct the MCMC PCA-Kameleon by simply substituting β-coefficients withρνjα(j), wherej is the selected eigendirection, andνj is the scaling factor associated to thej-th eigendirection.”
Kernel Mean Estimation and Stein's Effect (2013, neutral): “Moreover, the squared RKHS norm ‖·‖2 H can be decomposed as a sum of squared loss weighted by the eigenvalues γi (cf. [18, Appendix]).”
Domain Generalization via Invariant Feature Representation (2013, methodology): “We also showed that DICA generalizes KPCA and COIR, and is closely related to TCA.”
Kernel methods in machine learning (2007, neutral): “[119] applied the kernel trick to generalize principal component analysis and pointed out the (in retrospect obvious) fact that any algorithm which only uses the data via dot products can be generalized using kernels.”