Sparse Principal Component Analysis and Iterative Thresholding

By Zongming Ma. 2011

348 citations recorded by the source corpus.

Principal component analysis (PCA) is a classical dimension reduction method which projects data onto the principal subspace spanned by the leading eigenvectors of the covariance matrix. However, it behaves poorly when the number of features p is comparable to, or even much larger than, the sample size n. In this paper, we propose a new iterative thresholding approach for estimating principal subspaces in the setting where the leading eigenvectors are sparse. Under a spiked covariance model, we find that the new approach recovers the principal subspace and leading eigenvectors consistently, and even optimally, in a range of high-dimensional sparse settings. Simulated examples also demonstrate its competitive performance.

Subfields: Computational Mechanics, Statistics and Probability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

In Signal Processing

Citations in the corpus

Principal Components and Regularized Estimation of Factor Models (2017, background): “condition with respect to F for a given Λ is unaﬀected by the introduction of the linear constraints on Λ. Hence, the solution Fγ,τ = ZΛ(Λ′Λ+γIr)−1, ∀τ ≥ 0 (24) 18For SPC, see Jolliﬀee et al. (2003), Ma (2013), Shen and Huang (2008), and Zou et al. (2006). The SPC is in turn diﬀerent from the POET e”
Statistical inference for principal components of spiked covariance matrices (2020, methodology): “[56] Z.”
Robust Covariance Matrix Estimation via Matrix Depth (2015, background): “[27, 32, 46, 7], and is different from the notion of degree sparsity studied in [2, 5].”
A Sparse Singular Value Decomposition Method for High-Dimensional Data (2014, methodology): “This model has been used by Baik and Silverstein (2006), Paul (2007), Paul and Johnstone (2007), Johnstone and Lu (2009), and Ma (2013), among others.”
ECA: High-Dimensional Elliptical Component Analysis in Non-Gaussian Distributions (2013, methodology): “Ma (2013) and Yuan and Zhang (2013) proposed using modified versions of the power method to estimate eigenvectors and principal subspaces.”
Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance (2014, background): “See, for instance, [12, 19, 22, 26, 31] and the references cited therein.”
Robust covariance and scatter matrix estimation under Huber’s contamination model (2015, background): “[27, 32, 46, 7], and is different from the notion of degree sparsity studied in [2, 5].”
Truncated power method for sparse eigenvalue problems (2011, methodology): “Our method is also related to the Iterative Thresholding Sparse PCA (ITSPCA) method (Ma, 2013) which concentrates on recovering a sparse subspace of dimension m under the spike model.”
On estimation of the noise variance in high dimensional probabilistic principal component analysis (2013, background): “lication to determination criteria of Bai and Ng (2002) In econometrics, the assumption of additive white noise is reasonable for asset returns with low frequencies (e.g., monthly returns of stocks) (Ma, 2013). Bai and Ng (2002) proposes six criteria to estimate the number of PCs (or factors) under ”
Estimation of False Discovery Proportion with Unknown Dependence (2013, background): “to Weyl’s Theorem (Horn & Johnson, 1990) and the second result is called the sinTheorem (Davis & Kahan, 1970). They have been applied in sparse covariance matrix estimation (El Karoui, 2008; Ma, 2013). By Lemma 1, the consistency of eigenvectors and eigenvalues is directly associated with th”
Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees (2015, methodology): “We considered two approaches for initialization: (a) diagonal thresholding as described in the papers [39, 48], and (b) choosing F 0 to be the perturbed version”
An Overview on the Estimation of Large Covariance and Precision Matrices (2015, positive): “The final graph estimate Ĝ is obtained by either the “AND” or “OR” rule on combining the neighborhoods for all the N nodes. To estimate Θ, we also estimate the σ(2) j ’s using the fitted sum of squared residuals σ̂ 2 j = T −1 ∥∥Y∗j − Y∗\jαj ∥∥2 2 , then plug it into (9). In another work, Yuan (2010)”
Random matrix theory in statistics: A review (2014, methodology): “…of the submatrix corresponding to the selected coordinates, and proved consistency of this estimator assuming p grows at most polynomially with n. Ma (2013) developed iterative thresholding sparse PCA (ITSPCA), which is based on repeated filtering, thresholding and orthogonalization steps that…”