Regularized estimation of large covariance matrices

By Peter J. Bickel, Elizaveta Levina. 2008

1,448 citations recorded by the source corpus.

This paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance. We show that these estimates are consistent in the operator norm as long as (logp) 2 =n ! 0, and obtain explicit rates. The results are uniform over some fairly natural well-conditioned families of covariance matrices. We also introduce an analogue of the Gaussian white noise model and show that if the population covariance is embeddable in that model and well-conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix. The results can be extended to smooth versions of banding and to non-Gaussian distributions with su‐ciently short tails. A resampling approach is proposed for choosing the banding parameter in practice. This approach is illustrated numerically on both simulated and real data.

Subfields: Computer Networks and Communications, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/2a4a3c9b6aac9733e11c8860a5be1f2d32a617bd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Statistics and Probability

Citations in the corpus

Finite sample change point inference and identification for high‐dimensional mean vectors (2017, background): “…of Σ (or the inverse Σ−1) is possible under additional structural assumptions (such as sparsity or low-rankness) on the covariance matrix (Barigozzi et al., 2018; Bickel & Levina, 2008a,b; Cai & Zhou, 2012; Cai et al., 2010; Chen et al., 2013), which may not hold in practical applications.”
Forward Regression for Ultra-High Dimensional Variable Screening (2009, background): “Then, by Conditions (C1), (C2), and Lemma A3 in Bickel and Levina (2008), there exists constants C1 > 0 and C2 > 0, such that that P (|σ̂j1j2−σj1j2| > 2) ≤ C1 exp(−C2n2).”
The Masked Sample Covariance Estimator: An Analysis via Matrix Concentration Inequalities (2011, background): “Over the last few years, a number of papers, including [4, 5, 7, 10, 12, 32], have focused on the situation where the covariance matrix is sparse or nearly so.”
Modeling and Forecasting Large Realized Covariance Matrices and Portfolio Choice (2017, neutral): “Several other papers have dealt with the estimation of vast covariance matrices, e.g. Bickel and Levina (2008), Levina et al. (2008), Fan et al. (2008), Wang and Zou (2010), Fan et al. (2011), Fan et al. (2012), Fan et al. (2012), Hautsch et al. (2012), Fan et al. (2013).”
Random matrix theory in statistics: A review (2014, background): “Thresholding individual entries of the sample covariance matrix is an effective approach when the covariance matrix is sparse, and with proper choice of thresholds it has been shown to be consistent in the operator norm by Bickel and Levina (2008b) and El Karoui (2008a).”
Envelopes and partial least squares regression (2013, background): “(See Aldrich (2005) for a review and a historical perspective.)”
High-dimensional analysis of semidefinite relaxations for sparse principal components (2008, methodology): “ticular on the behavior of various ℓ1-based relaxation methods. Other work has tackled the problem of estimating sparse covariance matrices in the high-dimensional setting, using thresholding methods [3, 12] as well as ℓ1-regularization methods [8, 39]. A related problem—and the primary focus of thi”
Sequential Covariance-Matrix Estimation with Application to Mitigating Catastrophic Forgetting (2015, background): “An extensive body of literature concerning improved estimators in such situations exist [10], [11].”
Multitask Quantile Regression under the Transnormal Model (2016, methodology): “With guaranteed positive definiteness, banded Cholesky decomposition regularization receives much attention; for example, Wu and Pourahmadi (2003); Huang et al. (2006); Bickel and Levina (2008a); and Levina, Rothman, and Zhu (2008).”
Factor models and variable selection in high-dimensional regression analysis (2011, background): “We want to emphasize that this situation is very different from the setup which is usually considered in the literature on the analysis of high-dimensional covariance matrices; see, for example, Bickel and Levina (2008). It is then assumed that the variables of interest are only weakly correlated an”
A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation (2011, methodology): “The following lemma shows that solving the optimization problem (1) is equivalent to solving the p optimization problems (3).”
Covariance Estimation: The GLM and Regularization Perspectives (2011, methodology): “However, just as in banding, Bickel and Levina (2008b) have established the consistency of the threshold estimator in the operator norm, uniformly over the class of matrices that satisfy a notion of sparsity, provided that log pn → 0.”
Testing mutual independence in high dimension via distance covariance (2016, background): “Bickel and Levina (2008) and Qiu and Chen (2012) (the test statistic values are shown in Fig.”