Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance

By Vladimir Koltchinskii, Karim Lounici. 2014

103 citations recorded by the source corpus.

Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables with zero mean and covariance operator $\Sigma={\mathbb E}(X\otimes X)$ taking values in a separable Hilbert space ${\mathbb H}.$ Let $$ {\bf r}(\Sigma):=\frac{{\rm tr}(\Sigma)}{\|\Sigma\|_{\infty}} $$ be the effective rank of $\Sigma,$ ${\rm tr}(\Sigma)$ being the trace of $\Sigma$ and $\|\Sigma\|_{\infty}$ being its operator norm. Let $$\hat \Sigma_n:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ The paper deals with a problem of estimation of spectral projectors of the covariance operator $\Sigma$ by their empirical counterparts, the spectral projectors of $\hat \Sigma_n$ (empirical spectral projectors). The focus is on the problems where both the sample size $n$ and the effective rank ${\bf r}(\Sigma)$ are large. This framework includes and generalizes well known high-dimensional spiked covariance models. Given a spectral projector $P_r$ corresponding to an eigenvalue $\mu_r$ of covariance operator $\Sigma$ and its empirical counterpart $\hat P_r,$ we derive sharp concentration bounds for bilinear forms of empirical spectral projector $\hat P_r$ in terms of sample size $n$ and effective dimension ${\bf r}(\Sigma).$ Building upon these concentration bounds, we prove the asymptotic normality of bilinear forms of random operators $\hat P_r -{\mathbb E}\hat P_r$ under the assumptions that $n\to \infty$ and ${\bf r}(\Sigma)=o(n).$ In a special case of eigenvalues of multiplicity one, these results are rephrased as concentration bounds and asymptotic normality for linear forms of empirical eigenvectors. Other results include bounds on the bias ${\mathbb E}\hat P_r-P_r$ and a method of bias reduction as well as a discussion of possible applications to statistical inference in high-dimensional principal component analysis.

Subfields: Geometry and Topology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d2a919e77546209d9328f8392749a18334d366ca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Geometry and Topology

In Statistics and Probability

Citations in the corpus

Normal approximation and concentration of spectral projectors of sample covariance (2015, methodology): “We will need a couple of lemmas proved in [13] (see Lemmas 1 and 4 therein).”
Spectral Method and Regularized MLE Are Both Optimal for Top-$K$ Ranking (2017, unknown): “In particular, `∞ perturbation bounds for eigenvectors of symmetric matrices (Koltchinskii and Lounici, 2016; Fan, Wang and Zhong, 2016; Eldridge, Belkin and Wang, 2017; Abbe et al., 2017) and singular vectors of general matrices (Koltchinskii and Xia, 2016) have been studied.”
Nonasymptotic upper bounds for the reconstruction error of PCA (2016, background): “In view of high-dimensional applications, empirical spectral pro- jectors (such as P̂6d) are often studied when the distance to the population spectral projectors is measured in operator or Hilbert-Schmidt norm, see e.g. Zwald and Blanchard [32], Nadler [25], Mas and Ruymgaart [23], and Koltchinskii”