Adaptive Thresholding for Sparse Covariance Matrix Estimation

By Tommaso Cai, Weidong Liu. 2011

564 citations recorded by the source corpus.

In this article we consider estimation of sparse covariance matrices and propose a thresholding procedure that is adaptive to the variability of individual entries. The estimators are fully data-driven and demonstrate excellent performance both theoretically and numerically. It is shown that the estimators adaptively achieve the optimal rate of convergence over a large class of sparse covariance matrices under the spectral norm. In contrast, the commonly used universal thresholding estimators are shown to be suboptimal over the same parameter spaces. Support recovery is discussed as well. The adaptive thresholding estimators are easy to implement. The numerical performance of the estimators is studied using both simulated and real data. Simulation results demonstrate that the adaptive thresholding estimators uniformly outperform the universal thresholding estimators. The method is also illustrated in an analysis on a dataset from a small round blue-cell tumor microarray experiment. A supplement to this article presenting additional technical proofs is available online.

Subfields: Computational Mechanics, Statistics and Probability, Signal Processing

View Paper: https://www.semanticscholar.org/paper/456fcfa16a09ae96d4181ef7be0157fbb24eda38

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Tommaso Cai
2. Yuan Liao
3. Harrison H. Zhou
4. Jianqing Fan
5. Zhao Ren
6. Lingzhou Xue
7. Hui Zou
8. Jushan Bai

In Statistics and Probability

1. Tommaso Cai
2. Jianqing Fan
3. Yuan Liao
4. Clifford Lam
5. Harrison H. Zhou
6. D. Xiu
7. Lingzhou Xue
8. Zhao Ren

In Signal Processing

1. Jianqing Fan
2. Yuan Liao
3. Tommaso Cai
4. D. Xiu
5. Yacine Aït-Sahalia
6. Zhao Ren
7. Harrison H. Zhou
8. Lingzhou Xue

Citations in the corpus

• Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation (2012, methodology): “Cai and Liu (2011) introduced an adaptive thresholding procedure for estimating sparse covariance matrices that automatically adjusts to the variability of individual entries.”
• MINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER ℓ1-NORM (2012, background): “See, for example, Wu and Pourahmadi (2003), Zou, Hastie, and Tibshirani (2006), Bickel and Levina (2008a, b), El Karoui (2008), Lam and Fan (2009), Johnstone and Lu (2009), Cai, Zhang, and Zhou (2010), and Cai and Liu (2011).”
• Efficient Estimation of Approximate Factor Models (2012, methodology): “Define a general thresh- olding function sij(z) : R → R as in Rothman et al. (2009) and Cai and Liu (2011) with an entry-dependent threshold τij such that: (i) sij(z) = 0 if |z| < τij ; (ii) |sij(z)− z| ≤ τij .”
• Large Covariance Estimation by Thresholding Principal Orthogonal Complements (2011, unknown): “This section compares POET with direct thresholding on the sample covariance matrix without taking out common factors (Rothman et al. 2009, Cai and Liu 2011.”
• Incorporating Global Industrial Classification Standard Into Portfolio Allocation: A Simple Factor-Based Large Covariance Matrix Estimator With High-Frequency Data (2016, unknown): “Alternatively, the thresholding approach is also very popular, including Bickel and Levina (2008a, 2008b) and Cai and Liu (2011), etc. Fan and Liu (2013) and Zhou, Cai, and Ren (2014) presented a very comprehensive review of this literature.”
• Covariance Regression Analysis (2017, unknown): “In the last few years, various sparsity constraints have been imposed on either (Huang et al. 2006; Bickel and Levina 2008a, 2008b; Cai and Liu 2011; Leng and Li 2011), its inverse −1 (Dempster 1972; Friedman, Hastie, and Tibshirani 2008; Cai, Liu, and Luo 2011), or its eigenvalues (Johnstone and…”
• Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity (2015, methodology): “We can use the plug-in estimator of the thresholding estimators proposed in [4, 11].”
• Fundamental Limits of Online and Distributed Algorithms for Statistical Learning and Estimation (2013, background): “A special case is a simple and natural sparse covariance estimation problem ([16, 19]), where we assume that all covariates are uncorrelated ( E [ x i x j ] = 0 ) except for a unique correlated pair of covariates ( i ∗ , j ∗ ) which we need to detect.”
• Random matrix theory in statistics: A review (2014, methodology): “Cai and Liu (2011a) proposed an adaptive thresholding rule and showed that it attains the minimax rate under the spectral norm over a class of covariance matrices with uniform lq constraint on the rows.”
• Positive Definite $\ell_1$ Penalized Estimation of Large Covariance Matrices (2012, methodology): “For all three regularized estimators, the thresholding parameter was chosen by 5-fold cross-validation (Bickel and Levina, 2008b; Rothman et al., 2009; Cai and Liu, 2011).”
• Inverse Moment Methods for Sufficient Forecasting Using High-Dimensional Predictors (2016, unknown): “The error covariance matrix Σu can be estimated by thresholding the sample covariance matrix of the estimated residual xt − B̂f̂t, denoted by Σ̂u = (σ̂ ij)p×p, as in Cai and Liu (2011), Xue et al. (2012), Fan et al. (2013) and Fan et al.”
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression (2018, background): “The tail bounds presented in this section also play a central role in sparse covariance matrix estimation as shown in Bickel and Levina (2008) and Cai and Liu (2011).”
• Multivariate-Sign-Based High-Dimensional Tests for the Two-Sample Location Problem (2016, background): “…has been a growing body of research in large-scale covariance matrix estimation under certain assumptions of sparseness (Bickel and Levina 2008; Cai and Liu 2011), obtaining a sparse estimator of scatter matrix in the robust context appears to be even more complicated and has not been…”
• The spectral norm of random inner-product kernel matrices (2015, background): “Optimal rates of convergence under various matrix norms and sparsity models were established in [15, 14], and generalizations to other thresholding functions and to “adaptive” entryspecific thresholds were studied respectively in [46] and [11].”
• HIGH DIMENSIONAL COVARIANCE MATRIX ESTIMATION IN APPROXIMATE FACTOR MODELS (2011, unknown): “Moreover, this paper uses the hard-thresholding technique, which takes the form of b | | Rothman, Levina and Zhu (2009) and Cai and Liu (2011) studied a more general thresholding function of Antoniadis and Fan (2001), which admits the form σ ij ( θ ij ) = s ( σ ij ), and also allows for soft-thresho”
• Using Principal Component Analysis to Estimate a High Dimensional Factor Model with High-Frequency Data (2017, methodology): “Alternative methods rely on various forms of thresholding (Bickel and Levina (2008a), Bickel and Levina (2008b), Cai and Liu (2011), Fryzlewicz (2013), and Zhou et al. (2014)) whereas the estimator in Fan et al. (2013) is designed for latent factor models.”
• Substitution principle for CLT of linear spectral statistics of high-dimensional sample covariance matrices with applications to hypothesis testing (2014, background): “Bickel and Levina [7, 8], Cai and Liu [10] and the references therein).”
• An Overview on the Estimation of Large Covariance and Precision Matrices (2015, methodology): “For example, using the simple thresholding, we can define the adaptive thresholding estimator (Cai and Liu, 2011): Σ̂ = (σ̂ij)p×p, σ̂ij = sij, if i = jsij1{|sij|/SE(sij) > ωT}, if i 6= j , (4) where SE(sij) is the estimated standard error of sij.”