Normal approximation and concentration of spectral projectors of sample covariance

By Vladimir Koltchinskii, Karim Lounici. 2015

83 citations recorded by the source corpus.

Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables in a separable Hilbert space ${\mathbb H}$ with zero mean and covariance operator $\Sigma={\mathbb E}(X\otimes X),$ and let $\hat \Sigma:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ Denote by $P_r$ the spectral projector of $\Sigma$ corresponding to its $r$-th eigenvalue $\mu_r$ and by $\hat P_r$ the empirical counterpart of $P_r.$ The main goal of the paper is to obtain tight bounds on $$ \sup_{x\in {\mathbb R}} \left|{\mathbb P}\left\{\frac{\|\hat P_r-P_r\|_2^2-{\mathbb E}\|\hat P_r-P_r\|_2^2}{{\rm Var}^{1/2}(\|\hat P_r-P_r\|_2^2)}\leq x\right\}-\Phi(x)\right|, $$ where $\|\cdot\|_2$ denotes the Hilbert--Schmidt norm and $\Phi$ is the standard normal distribution function. Such accuracy of normal approximation of the distribution of squared Hilbert--Schmidt error is characterized in terms of so called effective rank of $\Sigma$ defined as ${\bf r}(\Sigma)=\frac{{\rm tr}(\Sigma)}{\|\Sigma\|_{\infty}},$ where ${\rm tr}(\Sigma)$ is the trace of $\Sigma$ and $\|\Sigma\|_{\infty}$ is its operator norm, as well as another parameter characterizing the size of ${\rm Var}(\|\hat P_r-P_r\|_2^2).$ Other results include non-asymptotic bounds and asymptotic representations for the mean squared Hilbert--Schmidt norm error ${\mathbb E}\|\hat P_r-P_r\|_2^2$ and the variance ${\rm Var}(\|\hat P_r-P_r\|_2^2),$ and concentration inequalities for $\|\hat P_r-P_r\|_2^2$ around its expectation.

Subfields: Mathematical Physics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/650f3cb9045fabd5bdc4f6051f0bd3d235d19fc6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Mathematical Physics

In Statistics and Probability

Citations in the corpus

Singular vector and singular subspace distribution for the matrix denoising model (2018, background): “Considering the Gaussian data samples xi ' N (0,Σ), i = 1, 2, · · · , n and X = (xi), under the assumption that the order of TrΣ ‖Σ‖ is much smaller than n, in [45, 46, 47], the authors prove that the eigenvectors of XX∗ are asymptotically normally distributed, whose variance depends the eigenvector”
Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance (2014, methodology): “ion 6, we brieﬂy discuss potential applications of these results, in particular, to the problem of support recovery of the eigenvector of interest as well as sparse PCA estimation. In a related paper [18], we obtained an asymptotic formula for the Hilbert– Schmidt norm risk EkPˆr− Prk2 2 of empirica”
Nonasymptotic upper bounds for the reconstruction error of PCA (2016, background): “In view of high-dimensional applications, empirical spectral pro- jectors (such as P̂6d) are often studied when the distance to the population spectral projectors is measured in operator or Hilbert-Schmidt norm, see e.g. Zwald and Blanchard [32], Nadler [25], Mas and Ruymgaart [23], and Koltchinskii”