Covariance regularization by thresholding

By Peter J. Bickel, Elizaveta Levina. 2009

1,340 citations recorded by the source corpus.

This paper considers regularizing a covariance matrix of $p$ variables estimated from $n$ observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and $(\log p)/n\to0$, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general cross-validation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/f35dfefdd5c44c5e717e17bb72c273b22b0878f1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

FAST CONVERGENCE RATES IN ESTIMATING LARGE VOLATILITY MATRICES USING HIGH-FREQUENCY FINANCIAL DATA (2013, methodology): “For estimating the large sparse covariance matrix of Gaussian data, Bickel and Levina (2008) constructed a threshold estimator that can achieve con- vergence rate π(p) [ n−1/2 log1/2 p ]1−δ , and Cai and Zhou (2012, 2013) showed that such a convergence rate is optimal.”
Invariant Scattering Convolution Networks (2012, background): “ number of training samples per class. Indeed, if there are few training samples per class then variance terms dominate bias errors when estimating off-diagonal covariance coefﬁcients between classes [4]. An afﬁne space approximation classiﬁer can also be interpreted as a robust quadratic discrimina”
Operator norm consistent estimation of large-dimensional sparse covariance matrices (2008, background): “ we can estimate consistently. Finally, we note that at the same time as we were researching these questions and independently, similar questions were approached from a very diﬀerent point of view by [8]. 2. Sparse matrices: concepts and deﬁnitions. One conceptual diﬃculty of thisproblemis to deﬁnea”
Latent variable graphical model selection via convex optimization (2010, neutral): “, times series), while “permutation-invariant” methods that use thresholding are useful when there is no natural variable ordering [13], [3].”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “…contains other commonly considered classes of sparse covariance matrices in the literature, including an q ball assumption maxi ∑p j=1 |σij | q ≤ sn,p in Bickel and Levina (2008b), and a weak q ball assumption max1≤j≤p {∣∣σj[k]∣∣q} ≤ sn,p/k for each integer k in Cai and Zhou (2012a)…”
Forecasting volatility and co-volatility of crude oil and gold futures: Effects of leverage, jumps, spillovers, and geopolitical risks (2020, methodology): “As in Asai and McAleer (2017), we also apply thresholding of Bickel and Levina (2008) to guarantee the positive (semi-)de(cid:12)niteness of the estimators.”
Analysis of elliptical copula correlation factor model with Kendall's tau (2013, methodology): “Possibilities of generic estimators T̂ ′ of T include regularized estimators such as thresholding [2, 6] or tapering [5] estimator.”
A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation (2011, methodology): “To solve (19), we follow the primal dual interior method approach (see, e.g., Boyd and Vandenberghe 2004).”
Detection of correlations (2011, background): “By Ery Arias-Castro1, Sébastien Bubeck and Gábor Lugosi2”
Modelling matrix time series via a tensor CP-decomposition (2021, positive): “In the spirit of Bickel and Levina (2008), we select Σ̂k defined as follows for the estimate of ΣY,ξ(k): Σ̂k = Tδ1{Σ̂Y,ξ(k)} , (15) where Tδ1(·) is a threshold operator Tδ1(W) = {wi,jI(|wi,j | ≥ δ1)}m1×m2 for any matrix W = (wi,j)m1×m2 with the threshold level δ1 ≥ 0.”
High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation (2010, methodology): “We refer the interested reader to Johnstone (2001), El Karoui (2007, 2008, 2009a), Bickel and Levina (2008a), Rothman et al. (2008) for a partial introduction to these problems. We wish with this study to make clear that the “large n, large p” character of the problem has an important impact of the ”
An Overview on the Estimation of Large Covariance and Precision Matrices (2015, positive): “1 Thresholding estimation One of the most convenient methods to estimate sparse covariance matrices is the thresholding, which sets small estimated elements to zero (Bickel and Levina, 2008).”
Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications (2016, background): “d is larger or even much larger than n) have attracted a lot of research attentions such as covariance matrix estimation [9, 10, 30, 20], graphical models [28, 74, 12], discriminant analysis [50], factor models [31, 45], among many others.”