Principal Component Analysis
By Ian T. Jolliffe, Michael J. Greenacre, Patrick J. F. Groenen, Trevor J. Hastie, Alfonso Iodice D'Enza, Angelos I. Markos, Elena Tuzhilina. 2003
27,017 citations recorded by the source corpus.
Principal component analysis is a versatile statistical method for reducing a cases-by-variables data table to its essential features, called principal components. Principal components are a few linear combinations of the original variables that maximally explain the variance of all the variables. In the process, the method provides an approximation of the original data table using only these few major components. This Primer presents a comprehensive review of the method’s definition and geometry, as well as the interpretation of its numerical and graphical results. The main graphical result is often in the form of a biplot, using the major components to map the cases and adding the original variables to support the distance interpretation of the cases’ positions. Variants of the method are also treated, such as the analysis of grouped data, as well as the analysis of categorical data, known as correspondence analysis. Also described and illustrated are the latest innovative applications of principal component analysis: for estimating missing values in huge data matrices, sparse component estimation, and the analysis of images, shapes and functions. Supplementary material includes video animations and computer scripts in the R environment. Principal component analysis is a multivariate statistical method that reduces a large number of variables into fewer variables, called principal components. This Primer describes how the method can be used for data analysis, explaining the mathematical background, analytical workflows, how to interpret a biplot and variants of the method.
Subfields: Analytical Chemistry, Computer Vision and Pattern Recognition, Statistics and Probability, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Analytical Chemistry
- Joaquim Jaumot
- A. de Juan
- Pedram Ghamisi
- Romà Tauler
- Antonio J. Plaza
- Jon Atli Benediktsson
- Jocelyn Chanussot
- Javier Plaza
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Chunhua Shen
- Yi Yang
In Statistics and Probability
- Marc Hallin
- Thomas Verdebout
- Davy Paindaveine
- Sham M. Kakade
- Prateek Jain
- Jun Zhu
- Julie Josse
- Praneeth Netrapalli
In Representation & Self-Supervised Learning
- Dacheng Tao
- Jun Zhu
- Stephan Günnemann
- Philip S. Yu
- Paolo Favaro
- Xiang Ren
- Yoshua Bengio
- Tao A. Xiang
Citations in the corpus
- Remote Sensing Image Scene Classification: Benchmark and State of the Art (2017, neutral): “…track field, harbor, industrial area, intersection, island, lake, meadow, medium residential, mobile home park, mountain, overpass, palace, parking lot, railway, railway station, rectangular farmland, river, roundabout, runway, sea ice, ship, snowberg, sparse residential, stadium, storage tank,…”
- GAN Inversion: A Survey (2021, methodology): “PCA enforces a weak orthogonality constraint, resulting in a distributed representation that uses cancellations to capture global variability [191], [192], while VQ uses a strong winner-take-all constraint that leads to clustering the data into mutually exclusive prototypes [193].”
- Fast Edge Detection Using Structured Forests (2014, positive): “Finally, Principal Component Analysis (PCA) [25] can be used to further reduce the dimensionality of Z .”
- How algorithmic confounding in recommendation systems increases homogeneity and decreases utility (2017, background): “Other latent factor methods, such as principal component analysis (PCA) [29] and latent Dirichlet allocation (LDA) [7], similarly frame user and item representations in terms of a low dimension K of hidden factors.”
- Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, neutral): “In traditional machine learning, approaches such as PCA [44], ICA [42] and spectrum analysis [100] are proposed to discover disentangled components of data.”
- Deep Metric Learning via Lifted Structured Feature Embedding (2015, neutral): “In this regard, metric learning [13, 39, 34] and dimensionality reduction [18, 7, 29, 2] techniques aim at learning semantic distance measures and embeddings such that similar input objects are mapped to nearby points on a manifold and dissimilar objects are mapped apart from each other.”
- Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks (2016, methodology): “For (1), we first embed all of the images in a class in a two-dimensional space via PCA (Person, 1901; Jolliffe, 2002) and t-SNE (Van der Maaten et al., 2008), and then perform k -means clustering (MacQueen et al., 1967) to find k types of images (Fig.”
- Examining Gender Bias in Languages with Grammatical Gender (2019, methodology): “Therefore, instead of applying PCA, we learn the grammatical gender direction (cid:126)d g by Linear Discriminant Analysis (LDA) (Fisher, 1936), which is a standard approach for supervised dimension reduction.”
- Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks (2021, methodology): “Figure 3 illustrates the 2D vector projections of task embeddings using PCA (Wold et al., 1987 Prior work also proposed meta-learning or Bayesian approaches to generate softmax layer parameters for new settings (Bansal et al., 2020; Ponti et al., 2020).”
- Optimization Models for Machine Learning: A Survey (2019, background): “1 Principal Components Principal Components Analysis (PCA) [128] constructs features with large variance based on the original set of features.”
- Interactive Grounded Language Acquisition and Generalization in a 2D World (2018, methodology): “Each word context vector is projected down to a space of 50 dimensions using PCA (Jolliffe, 1986), after which we use t-SNE (van der Maaten & Hinton, 2008; Ulyanov, 2016) to further reduce the dimensionality to 2.”
- The Indian Buffet Process: An Introduction and Review (2011, neutral): “…(e.g., Ueda and Saito, 2003), allow each feature to take on a continuous value, representing datapoints with locations in a latent space (e.g., Jolliffe, 1986), or define a factorial model, in which each feature takes on one of a discrete set of values (e.g., Zemel and Hinton, 1994;…”
- A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security (2018, methodology): “In this subsection, we discuss the common ML algorithms (i.e. decision trees (DT), support vector machines (SVM), Bayesian algorithms, k-nearest neighbour (KNN), random forest (RF), association rule (AR) algorithms, ensemble learning, k-means clustering and principal component analysis (PCA)) and th”
- Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams (2012, methodology): “BSFA techniques rely on batch principal component analysis (PCA; Jolliffe (1986)).”