On a Connection between Kernel PCA and Metric Multidimensional Scaling

By Christopher K. I. Williams. 2004

250 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

On the Schoenberg Transformations in Data Analysis: Theory and Illustrations (2010, background): “…the object features themselves) corresponds an equally eﬃcient “distance trick”, stating that Euclidean distances themselves (and not their underlying coordinates) permit to express all the real quantities of interest, as in (5), (6), or Section 5; see also Sch¨olkopf (2000) and Williams (2002).”
Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, background): “Multidimensional scaling (MDS) is a classic form of dimensionality reduction Williams (2001).”
Spectral Methods for Dimensionality Reduction (2006, unknown): “We can thus interpret kernel PCA as a nonlinear version of MDS that results from substituting generalized dot products in feature space for Euclidean dot products in input space [Williams, 2001].”
Dimensionality Reduction: A Comparative Review (2008, methodology): “First, traditional PCA is identical to performing classical scaling and to performing Kernel PCA with a linear kernel, due to the relation between the eigenvectors of the covariance matrix and the doublecentered squared Euclidean distance matrix [143], which is in turn equal to the Gram matrix.”
Kernel Methods in Finance (2008, methodology): “Metric MDS can be categorised as a kernel method because it can be interpreted as a form of kernel MDS or kernel PCA, respectively (Williams, 2001).”
Multi-Task Learning using Generalized t Process (2010, background): “Then, according to the relationship between a distance matrix and a kernel matrix as in multidimensional scaling (Williams, 2001), we can get Ψ = HΨ′H where H = Im − 1 m1m1 T m is the m × m centering matrix and the (i, j)th element of Ψ′ is Ψij = ( 1 ni ∑ni k=1 (xk) )( 1 nj ∑nj l=1 (xl ) ) =”
A Spectral Approach to Gradient Estimation for Implicit Distributions (2018, methodology): “Connection to Manifold-modeling Dimension Reduction Methods It has been pointed out in previous works (Williams, 2001; Ham et al., 2004; Bengio et al., 2004a;b) that many successful manifold-modeling dimen-sion reduction methods (e.g., MDS, LLE, Laplacian eigen-maps, and Spectral clustering) can be…”
Learning Eigenfunctions Links Spectral Embedding and Kernel PCA (2004, methodology): “Note also that Williams (2001) makes a connection between kernel PCA and metric MDS, remarking that kernel PCA is a form of MDS when the kernel is isotropic.”
Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks? (2017, background): “ similarity matching (Pehlevan et al., 2015). This objective, borrowed from MultiDimensional Scaling (MDS), minimizes the mismatch between the similarity of 3 inputs and outputs (Mardia et al., 1980; Williams, 2001; Cox and Cox, 2000): PSP : min Y∈Rk×T 1 T2 X⊤X−Y⊤Y 2 F . (2) Here, similarity is quan”
Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models (2005, background): “There are strong connections between MDS and kernel PCA (Schölkopf et al., 1998), some of which are formalised in Williams (2001).”
Learning Distance Metrics with Contextual Constraints for Image Retrieval (2006, methodology): “…literature, the kernel trick has been successfully applied on many linear analysis techniques, such as Kernel Principal Component Analysis (PCA) [19], Kernel Fisher Discriminant Analysis [10, 12], Support Vector Machines [17], Kernel Independent Component Analysis [1], etc. Similar to these…”