Operator norm consistent estimation of large-dimensional sparse covariance matrices

By Noureddine El Karoui. 2008

414 citations recorded by the source corpus.

Estimating covariance matrices is a problem of fundamental importance in multivariate statistics. In practice it is increasingly frequent to work with data matrices X of dimension n x p, where p and n are both large. Results from random matrix theory show very clearly that in this setting, standard estimators like the sample covariance matrix perform in general very poorly. In this "large n, large p" setting, it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0, and hence this matrix is sparse. We develop an estimator to handle this situation. The estimator is shown to be consistent in operator norm, when, for instance, we have p ? n as n → oo. In other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero. This implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues. We also propose a notion of sparsity for matrices, that is, "compatible" with spectral analysis and is independent of the ordering of the variables.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/0cea65ebe445a4bd2c3d1a14899e9def6b644095

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation (2011, background): “El Karoui (2008) Tony Cai is Dorothy Silberberg Professor, Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104 (E-mail: tcai@wharton.upenn.edu).”
CONSISTENCY OF RESTRICTED MAXIMUM LIKELIHOOD ESTIMATORS OF PRINCIPAL COMPONENTS (2008, background): “Recently several approaches have been proposed for estimating large dimensional covariance matrices and their eigenvalues and eigenvectors under suitable sparsity assumptions on the population covariance, e.g. Bickel and Levina (2007, 2008) and El Karoui (2008).”
Estimation of (near) low-rank matrices with noise and high-dimensional scaling (2009, background): “els with sparsity constraints, including the problem of sparse linear regression [52, 16, 18, 40, 10], banded or sparse covariance matrices [7, 8, 19], sparse inverse covariance matrices [57, 24, 50, 46], sparse eigenstructure [30, 2, 44], and sparse regression matrices [43, 36, 56, 28].”
Distances and inference for covariance operators (2014, background): “The distance dL(· , ·) generalizes the matrix spectral norm often used in the finite-dimensional case (see, e.g., El Karoui, 2008).”
MINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER ℓ1-NORM (2012, background): “See, for example, Wu and Pourahmadi (2003), Zou, Hastie, and Tibshirani (2006), Bickel and Levina (2008a, b), El Karoui (2008), Lam and Fan (2009), Johnstone and Lu (2009), Cai, Zhang, and Zhou (2010), and Cai and Liu (2011).”
High-dimensional covariance matrix estimation with missing observations (2012, background): “An other popular approach assumes that the unknown covariance matrix is sparse,that is most of the entries are exactly or approximately zero and then proposes to perform either entrywise thresholding or tapering of the sample covariance matrix [3, 6, 8, 14, 31, 32].”
Time-Varying Risk Premium in Large Cross-Sectional Equity Data Sets (2016, methodology): “In the setting of Section 2, we do not follow rolling short-window regression approaches to account for time-variation (Fama and French (1997), Lewellen and Nagel (2006)); see also Cosemans et al.”
High Dimensional Inverse Covariance Matrix Estimation via Linear Programming (2010, background): “See, for example, Ledoit and Wolf (2004), Levina, Rothman and Zhu (2007), Deng and Yuan (2008), El Karoui (2008), Fan, Fan and Lv (2008), Ravikumar, Raskutti, Wainwright and Yu (2008), Ravikumar, Wainwright, Ruskutti c©2010 Ming Yuan. and Yu (2008), Rocha, Zhao and Yu (2008), Lam and Fan (2009),…”
Positive Definite $\ell_1$ Penalized Estimation of Large Covariance Matrices (2012, methodology): “…have been used to improve the sample covariance matrix estimator, including banding (Wu and Pourahmadi, 2003; Bickel and Levina, 2008a), tapering (Furrer and Bengtsson, 2007; Cai, Zhang, and Zhou, 2010) and thresholding (Bickel and Levina, 2008b; El Karoui, 2008; Rothman, Levina, and Zhu, 2009).”
Estimation of Simultaneously Sparse and Low Rank Matrices (2012, methodology): “These methods are readily adapted to matrix valued data and have been applied to covariance estimation (El Karoui, 2009; Bien & Tibshirani, 2010) and graphical model structure learning (Banerjee et al., 2007; Friedman et al., 2008).”
Random matrix theory in statistics: A review (2014, methodology): “El Karoui (2008b) used the Marčenko–Pastur equation (18) and employed a linear programming method to minimize the sup-norm difference between z and the expression on the righthand side with the Stieltjes transform of the dual sample covariance matrix as input.”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “A similar idea was applied in the practical implementation without theoretical justification in El Karoui (2008).”
Latent variable graphical model selection via convex optimization (2010, neutral): “, times series), while “permutation-invariant” methods that use thresholding are useful when there is no natural variable ordering [13], [3].”
VAST VOLATILITY MATRIX ESTIMATION FOR HIGH-FREQUENCY FINANCIAL DATA (2010, background): “It is widely known that Γ̂ is a poor estimator of Γ when both n and p are large [Johnstone (2001), Johnstone and Lu (2009), El Karoui (2007, 2008), Bickel and Levina (2008a, 2008b)].”