MINIMAX SPARSE PRINCIPAL SUBSPACE ESTIMATION IN HIGH DIMENSIONS

By Vincent Q. Vu, Jing Lei. 2012

209 citations recorded by the source corpus.

We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We prove optimal, non-asymptotic lower and upper bounds on the minimax subspace estimation error under two different, but related notions of lq subspace sparsity for 0 � q � 1. Our upper bounds apply to general classes of covariance matrices, and they show that lq constrained estimates can achieve optimal minimax rates without restrictive spiked covariance conditions. 1. Introduction. Principal components analysis (PCA) was introduced in the early 20th century (Pearson, 1901; Hotelling, 1933) and is arguably the most well known and widely used technique for dimension reduction. It is part of the mainstream statistical repertoire and is routinely used in numerous and diverse areas of application. However, contemporary applications often involve much higher-dimensional data than envisioned by the early developers of PCA. In such high-dimensional situations, where the number of variables p is of the same order or much larger than the number of observations n, serious difficulties emerge: standard PCA can produce inconsistent estimates of the principal directions of variation and lead to unreliable conclusions (Johnstone and Lu, 2009; Paul, 2007; Nadler, 2008). The principal directions of variation correspond to the eigenvectors of the covariance matrix, and in high-dimensions consistent estimation of the eigenvectors is generally not possible without additional assumptions about the covariance matrix or its eigenstructure. Much of the recent development in PCA has focused on methodology that applies the concept of sparsity to eigenvector estimation (some examples include Jolliffe, Trendafilov and Uddin,

Subfields: Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/48bc312e171b81bfd065ac8ec259b36985c06e63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Nonasymptotic upper bounds for the reconstruction error of PCA (2016, background): “in Mas and Ruymgaart [25] in a functional setting and in Vu and Lei [34] and Cai, Ma, and Wu [8] in a high-dimensional sparse setting.”
Singular vector and singular subspace distribution for the matrix denoising model (2018, background): “De-biased sparse PCA: Inference and testing for eigenstructure of large covariance matrices. arXiv:1801.10567, 2018.”
Robust covariance and scatter matrix estimation under Huber’s contamination model (2015, background): “[27, 32, 46, 7], and is different from the notion of degree sparsity studied in [2, 5].”
Principal component analysis for second-order stationary vector time series (2014, background): “, it is intuitively clear that the smaller this diﬀerence is, more diﬃcult the estimation for M(Aj) is. See a similar phenomenon in estimating the principal subspaces with independent observations in Vu and Lei (2013). (ii) As Σy(k) = AΣx(k)A′, the maximum block size S max = max 1≤j≤q pj and the spa”
ECA: High-Dimensional Elliptical Component Analysis in Non-Gaussian Distributions (2013, methodology): “Similar algorithms were also studied by Vu and Lei (2012), Lounici (2013), Vu and Lei (2013), and Cai, Ma, and Wu (2015).”
Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees (2015, background): “A standard convex relaxation is based on the nuclear norm |||M |||nuc : = ∑d j=1 σj(M), corresponding to the sum of the singular values of the matrix.”
Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures (2017, background): “…et al. (2012), regularizing principal components in Johnstone and Lu (2009), Zou et al. (2006), Cai et al. (2013, 2015), Fan et al. (2013), and Vu and Lei (2013), and penalized regression for precision matrix estimation, Gaussian graphical models, and differential networks in Meinshausen and…”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “…studied by several papers under the sparse principal component analysis setting, including Paul (2007), Nadler (2008), Johnstone and Lu (2009), Amini andWainwright (2009), Birnbaum et al. (2013), Ma (2013), Vu and Lei (2013), Cai et al. (2013), Berthet and Rigollet (2013) and Cai et al. (2015).”
Consistency of spectral clustering in stochastic block models (2013, background): “A.1 Proof of Lemma 5.1 By proposition 2.2 of Vu & Lei (2013), there exists a K dimensional orthogonal matrix Q such that b b Next we establish that ( I − U U T ) UU T ≤ 2 A − P γ n . b b If A − P ≤ γ n / 2, then by Davis-Kahan sin Θ theorem, we have A.2 Proof of Lemma 5.3 First by the deﬁnition of…”
Robust Covariance Matrix Estimation via Matrix Depth (2015, background): “[27, 32, 46, 7], and is different from the notion of degree sparsity studied in [2, 5].”