Learning Eigenfunctions Links Spectral Embedding and Kernel PCA

By Yoshua Bengio, Olivier Delalleau, Nicolas Le Roux, Jean-François Paiement, Pascal Vincent, Marie Ouimet. Neural Computation, 2004

342 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cfa15801bf4e7bf610d38dc86da62b83e2ddedcb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Inductive Hashing on Manifolds (2013, background): “Then the desired hash functions may be efficiently identified by binarizing the Nyström eigenfunctions [2] with the approximated affinity matrix Ŵ.”
Diffusion Maps, Reduction Coordinates, and Low Dimensional Representation of Stochastic Systems (2008, methodology): “This formula is known as the Nyström extension and has been widely used in many contexts, see for example [5, 2].”
Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators (2005, methodology): “To this end, and similar to the framework of [16], we introduc e a statistical model and assume that the data points {xi} are i.”
Local Manifold Learning-Based $k$ -Nearest-Neighbor for Hyperspectral Image Classification (2010, methodology): “The embedding can be obtained by a generalized eigenvalue problem: ( D − W ) y = λ Dy , which is equivalent (up to scaling) to the eigenvalue decomposition of the M , referred to as the normalized graph Laplacian matrix, deﬁned by [24], [27].”
Dimensionality Reduction: A Comparative Review (2008, methodology): “Third, the spectral techniques Kernel PCA, Isomap, LLE, and Laplacian Eigenmaps can all be viewed upon as special cases of the more general problem of learning eigenfunctions [12,48].”
Nonparametric Conditional Density Estimation in a High-Dimensional Regression Setting (2016, neutral): “One can show that the Nyström extension (Drineas and Mahoney 2005) ψ̂j (x) = √ n l̂j ∑n k=1 ψ̃j (xk)K(x, xk) is a consistent estimate of ψj (Bengio et al. 2004).”
Spectral Clustering and Multidimensional Scaling: A Unified View (2006, methodology): “In the last decade, those methods have been very extensively studied in the machine learning community (see e.g. Shawe-Taylor and Cristianini 2004, and references therein), in relationship to manifold learning and spectral clustering (Bengio et al. 2004).”
Non-Local Manifold Tangent Learning (2004, methodology): “shown in (Bengio et al., 2004), the corresponding data-dependent kernel can be defined as KD(x, xi) = − 1 2 (D(x, xi) 2 − 1 n ∑ j D(x, xj) 2 − D̄i + D̄) where D̄i = 1 n ∑ j D(xi, xj) 2 and D̄ = 1 n ∑”
The Curse of Highly Variable Functions for Local Kernel Machines (2005, background): “See (Bengio et al., 2004) to see that LLE, Isomap, Laplacian eigenmaps and other spectral manifold learning algorithms such as spectral clustering can be generalized to be written as in eq.”
Hashing with Graphs (2011, methodology): “We create the “out-of-sample” extension of the Anchor Graph Laplacian eigenvectors Y to their corresponding eigenfunctions using the Nyström method (Williams & Seeger, 2001)(Bengio et al., 2004).”
A survey of kernel and spectral methods for clustering (2008, methodology): “A direct connection between Kernel PCA and spectral methods has been shown [89,90].”
Unsupervised Object Discovery: A Comparison (2010, background): “In fact Laplacian eigenmaps and KPCA solve very closely related learning problems (Bengio et al. 2004).”