PCA CONSISTENCY IN HIGH DIMENSION, LOW SAMPLE SIZE CONTEXT

By Sungkyu Jung, James Stephen Marron. 2009

329 citations recorded by the source corpus.

Principal Component Analysis (PCA) is an important tool of dimension reduction especially when the dimension (or the number of variables) is very high. Asymptotic studies where the sample size is fixed, and the dimension grows (i.e. High Dimension, Low Sample Size (HDLSS)) are becoming increasingly relevant. We investigate the asymptotic behavior of the Principal Component (PC) directions. HDLSS asymptotics are used to study consistency, strong inconsistency and subspace consistency. We show that if the first few eigenvalues of a population covariance matrix are large enough compared to the others, then the corresponding estimated PC directions are consistent or converge to the appropriate subspace (subspace consistency) and most other PC directions are strongly inconsistent. Broad sets of sucient conditions for each of these cases are specified and the main theorem gives a catalogue of possible combinations. In preparation for these results, we show that the geometric representation of HDLSS data holds under general conditions, which includes a mixing condition and a broad range of sphericity measures of the covariance matrix.

Subfields: Artificial Intelligence, Statistics and Probability, Signal Processing

View Paper: https://www.semanticscholar.org/paper/9d35c09419186f525df9c577f57ea29845a5398a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianqing Fan
2. Tommaso Cai
3. Yuan Liao
4. Yihong Wu
5. Michael J. Kane
6. Peter Bhlmann
7. Fang Han
8. Kazuyoshi Yata

In Statistics and Probability

1. Jianqing Fan
2. Yuan Liao
3. Tommaso Cai
4. Jushan Bai
5. Han Liu
6. Fang Han
7. Lingzhou Xue
8. Clifford Lam

In Signal Processing

1. Tommaso Cai
2. Jianqing Fan
3. Yuan Liao
4. Han Liu
5. Fang Han
6. Jushan Bai
7. Yihong Wu
8. Weichen Wang

Citations in the corpus

• PCA Consistency for Non-Gaussian Data in High Dimension, Low Sample Size Context (2009, background): “Jung and Marron (2008) discussed subspace consistency as d →∞ for the sample eigenvectors corresponding to the population eigenvalues given by (b’) that is described”
• An Overview on the Estimation of Large Covariance and Precision Matrices (2015, unknown): “Such an approach has drawn growing attentions in the recent literature on high-dimensional PCA (e.g., Jung and Marron (2009); Shen et al. (2013a,b); Ahn et al. (2007)).”
• ECA: High-Dimensional Elliptical Component Analysis in Non-Gaussian Distributions (2013, methodology): “8; Witten et al., 2009; Journee et al., 2010; Zhang and El Ghaoui, 2011). The theoretical properties of these and related methods have been analyzed under both Gaussian and sub-Gaussian assumptions (Jung and Marron, 2009; Amini and Wainwright, 2009; Shen et al., 2011; Ma, 2013; Vu and Lei, 2012, 201”
• Sparse Principal Component Analysis and Iterative Thresholding (2011, result): “See [16] for similar results in the case where p → ∞ and n is fixed.”
• Two-sample tests for high-dimension, strongly spiked eigenvalue models (2016, background): “For eigenvalues of high-dimensional data, Jung and Marron (2009), Yata and Aoshima (2012, 2013b), Onatski (2012) and Fan, Liao, and Mincheva (2013) considered spiked models such as λ ij → ∞ as p → ∞ for j = 1 , ..., k i , with some positive integer k i .”
• Sparse PCA: Optimal rates and adaptive estimation (2012, background): “At various levels of rigor and generality, a series of papers [4, 9, 23, 26, 30, 39, 43] showed that the sample principal eigenvectors are no longer consistent estimates of their population counterparts.”
• Handbook of Big Data (2016, methodology): “• Work such as [8] in 2009 has investigated conditions under which PCA estimates are consistent for fixed n and p → ∞.”
• PROJECTED PRINCIPAL COMPONENT ANALYSIS IN FACTOR MODELS (2014, background): “Such a required balance between the sample size and dimensionality is common for high-dimensional factor analysis (e.g., Bai (2003), Stock and Watson (2002)) and in the recent literature for PCA (e.g., Jung and Marron (2009), Shen et al. (2013)).”
• Efficient Estimation of Approximate Factor Models (2012, background): “ et al. (2012) that estimating a covariance matrix with fast diverging eigenvalues is a challenging problem. Other works on large covariance estimation include Cai and Zhou (2012), Fan et al. (2008), Jung and Marron (2009), Witten, Tibshirani and Hastie (2009), Deng and Tsui (2010), Yuan (2010), Led”
• Large Covariance Estimation by Thresholding Principal Orthogonal Complements (2011, unknown): “In addition, due to (2.3), the signals of the first K eigenvalues are stronger than those of the spiked covariance model considered by Jung and Marron (2009) and Birnbaum et al. (2012).”
• Limiting laws for divergent spiked eigenvalues and largest nonspiked eigenvalue of sample covariance matrices (2017, background): “One can refer to [31] for more discussions about this model.”