A Tutorial on Principal Component Analysis
By Jonathon Shlens. arXiv.org, 2014
2,740 citations recorded by the source corpus.
Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but (sometimes) poorly understood. The goal of this paper is to dispel the magic behind this black box. This manuscript focuses on building a solid intuition for how and why principal component analysis works. This manuscript crystallizes this knowledge by deriving from simple intuitions, the mathematics behind PCA. This tutorial does not shy away from explaining the ideas informally, nor does it shy away from the mathematics. The hope is that by addressing both aspects, readers of all levels will be able to gain a better understanding of PCA as well as the when, the how and the why of applying this technique.
Subfields: Analytical Chemistry, Molecular Biology, Signal Processing
View Paper: https://www.semanticscholar.org/paper/562e7f497eff8363825abad8d0008a42ce00eb49
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Analytical Chemistry
- Arjun Nitin Bhagoji
- Peter Bhlmann
- Prateek Mittal
- Dawn Song
- Bo Li
- Diogo M. Camacho
- Roger Zimmermann
- Kalyanmoy Deb
In Molecular Biology
- James C. Costello
- Diogo M. Camacho
- Rani K. Powers
- K. M. Collins
- Arjun Nitin Bhagoji
- Peter Bhlmann
- Prateek Mittal
- Dawn Song
In Signal Processing
- Bo Li
- Peter Bhlmann
- M. J. van der Laan
- Michael J. Kane
- Prateek Mittal
- Dawn Song
- Arjun Nitin Bhagoji
- Diogo M. Camacho
Citations in the corpus
- Exploring the Space of Black-box Attacks on Deep Neural Networks (2017, positive): “Using query reduction, IGE-QR (PCA-k , logit) with k = 100 with 10 iterations takes just 5s per sample.”
- On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice (2020, methodology): “PCA works by calculating the covariance matrix of the data matrix to obtain the eigenvectors of the covariance matrix.”
- Financial Development and Foreign Direct Investment—The Moderating Role of Quality Institutions (2020, neutral): “PCA is a non–parametric [87] and multivariate technique [88] that can extract relevant information from a large dataset where observations are generally depicted with several correlated quantitative dependent variables.”
- Handbook of Big Data (2016, methodology): “Furthermore, the statistical procedure principal component analysis has a direct relationship to the singular value decomposition (SVD) [16,26,31].”
- Empirical low-dimensional manifolds in composition space (2012, methodology): “The PCA is a general methodology to reduce a complex 235 dataset to a lower dimension, which can retain as much as possible of the 236 variation to reveal the simplified underlying structure of the data [17, 18].”
- A comprehensive survey of anomaly detection techniques for high dimensional big data (2020, background): “merging all the relevant attributes, called principal components, before eliminating all other remaining attributes [84, 85].”
- A high-bias, low-variance introduction to Machine Learning for physicists (2018, neutral): “For a detailed introduction to PCA, see the tutorials by Shlens (Shlens, 2014) and Bishop (Bishop, 2006).”
- Interpolation-based reduced-order modelling for steady transonic flows via manifold learning (2014, methodology): “A powerful tool currently considered state-of-the-art for the order reduction of large-scale systems is proper orthogonal decomposition (POD) (Holmes, Lumley, and Berkooz 1996, Section 3), also referred to as principal component analysis (Shlens 2005).”
- Dimensionality Reduction as a Defense against Evasion Attacks on Machine Learning Classifiers (2017, methodology): “Common dimensionality reduction algorithms are PCA [27], random projections [36] and Kernel PCA [37].”
- Machine Learning in Aerodynamic Shape Optimization (2022, methodology): “Principal Component Analysis PCA is a mathematical algorithm that transforms high-dimensional data to low dimensions while maintaining the maximum variation in the data set [198, 199].”
- Next-Generation Machine Learning for Biological Networks. (2018, positive): “As an example, by plotting the first two principal components of a PCA, one can judge the relative distance (a metric of similarity) between samples on a simple two-dimensional plot summarizing information from hundreds or thousands of features (Abdi and Williams, 2010; Shlens, 2014).”
- Objective Reduction in Many-Objective Optimization: Linear and Nonlinear Algorithms (2013, methodology): “Several nonlinear dimensionality reduction methods, such as Kernel PCA [38] and Graph-based methods [39], exist for removing the higher order dependencies.”
- Integration of New Information in Memory: New Insights from a Complementary Learning Systems Perspective (2020, neutral): “See [27] for more details.”
- Enhancing robustness of machine learning systems via data transformations (2017, methodology): “Specifically, we consider linear dimensionality reduction techniques such as Principal Component Analysis which aim to project high-dimensional data to a lower-dimensional space while preserving the most useful variance of the data [44], [51].”
- Generation of a realistic 3D sand assembly using X‐ray micro‐computed tomography and spherical harmonic‐based principal component analysis (2017, background): “Principal component analysis (PCA), a convenient statistical tool, was then used to extract the major patterns of the morphological features from the SH database.”