Optimal rates of convergence for covariance matrix estimation

By Tommaso Cai, Cun-Hui Zhang, Harrison H. Zhou. 2010

504 citations recorded by the source corpus.

Covariance matrix plays a central role in multivariate statistical analysis. Significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices. However, a minimax theory has yet been developed. In this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and Frobenius norm. It is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation. The minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties. A key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound. The technical analysis requires new ideas that are quite different from those used in the more conventional function/sequence estimation problems.

Subfields: Computer Networks and Communications, Statistics and Probability, Signal Processing

View Paper: https://www.semanticscholar.org/paper/18f5d04231d22104e73c82b372eb4c627da544a0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Statistics and Probability

In Signal Processing

Citations in the corpus

Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries (2016, background): “Estimation of the covariance matrix with respect to this norm has been previously investigated in the literature, for instance, see [LF09], [CZZ10] and references therein; Frobenius norm is a natural choice when one wants to understand the effect of the rank of an unknown covariance matrix on the es”
Estimation of high-dimensional low-rank matrices (2009, background): “…of high-dimensional matrices has been recently studied by several authors in settings diﬀerent from the ours [cf., e.g., Meinshausen and B¨uhlmann (2006), Bickel and Levina (2008), Ravikumar et al. (2008), Amini and Wainwright (2009), Cai, Zhang and Zhou (2010) and the references cited therein].”
Asymptotic normality and optimalities in estimation of large Gaussian graphical models (2013, background): “Statist. 38 2118-2144. Cai, T. T. and Zhou, H. H. (2012). Optimal rates of convergence for sparse covariance matrix estimation. Ann. Statist. 40 2389–2420. Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Found. of Comput. Math. 9 717-772. Chandrasekaran, V., Parr”
Robust Covariance Matrix Estimation via Matrix Depth (2015, background): “Then, Lemma 2 of [6] implies that ‖Σ̂ − Σ‖op ≤ Cmaxu∈Uk |uT (Σ̂−Σ)u| for some constant C > 0.”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “These include the banding method in Wu and Pourahmadi (2009) and Bickel and Levina (2008a), tapering in Furrer and Bengtsson (2007) and Cai et al. (2010), thresholding in Bickel and Levina (2008b), El Karoui (2008) and Cai and Liu (2011a), penalized likelihood estimation in Huang et al. (2006),…”
Multitask Quantile Regression under the Transnormal Model (2016, background): “Banding (Bickel & Levina 2008a) and tapering (Cai et al. 2010) were proposed to estimate bandable covariance matrices. But they have no guarantee of positive definiteness in practice. With the appealing positive definiteness, banded Cholesky decomposition regularization receives much attention, for ”
Detection of correlations (2011, background): “By Ery Arias-Castro1, Sébastien Bubeck and Gábor Lugosi2”
High-dimensional covariance matrix estimation with missing observations (2012, background): “An other popular approach assumes that the unknown covariance matrix is sparse,that is most of the entries are exactly or approximately zero and then proposes to perform either entrywise thresholding or tapering of the sample covariance matrix [3, 6, 8, 14, 31, 32].”
Finite sample change point inference and identification for high‐dimensional mean vectors (2017, background): “Spectral norm consistency of Σ (or the inverse Σ) is possible under additional structural assumptions (such as sparsity or low-rankness) on the covariance matrix (Barigozzi et al., 2018; Bickel & Levina, 2008a,b; Cai & Zhou, 2012; Cai et al., 2010; Chen et al., 2013), which may not hold in practical”
Covariance Matrix Estimation under Total Positivity for Portfolio Selection* (2019, methodology): “Such estimators, which are equivalent to selecting the covariance matrix closest to S in Frobenius norm subject to the covariance matrix lying within a speciﬁed L 1 ball, were proven to be minimax optimal with respect to the Frobenius norm and spectral norm loss [CZZ10].”
Analysis of elliptical copula correlation factor model with Kendall's tau (2013, methodology): “Possibilities of generic estimators T̂ ′ of T include regularized estimators such as thresholding [2, 6] or tapering [5] estimator.”