Asymptotic normality and optimalities in estimation of large Gaussian graphical models

By Zhao Ren, Tingni Sun, Cun-Hui Zhang, Harrison H. Zhou. 2013

259 citations recorded by the source corpus.

The Gaussian graphical model, a popular paradigm for studying relationship among variables in a wide range of applications, has attracted great attention in recent years. This paper considers a fundamental question: When is it possible to estimate low-dimensional parameters at parametric square-root rate in a large Gaussian graphical model? A novel regression approach is proposed to obtain asymptotically efficient estimation of each entry of a precision matrix under a sparseness condition relative to the sample size. When the precision matrix is not sufficiently sparse, or equivalently the sample size is not sufficiently large, a lower bound is established to show that it is no longer possible to achieve the parametric rate in the estimation of each entry. This lower bound result, which provides an answer to the delicate sample size question, is established with a novel construction of a subset of sparse precision matrices in an application of Le Cam's lemma. Moreover, the proposed estimator is proven to have optimal convergence rate when the parametric rate cannot be achieved, under a minimal sample requirement. The proposed estimator is applied to test the presence of an edge in the Gaussian graphical model or to recover the support of the entire model, to obtain adaptive rate-optimal estimation of the entire precision matrix as measured by the matrix $\ell_q$ operator norm and to make inference in latent variables in the graphical model. All of this is achieved under a sparsity condition on the precision matrix and a side condition on the range of its spectrum. This significantly relaxes the commonly imposed uniform signal strength condition on the precision matrix, irrepresentability condition on the Hessian tensor operator of the covariance matrix or the $\ell_1$ constraint on the precision matrix. Numerical results confirm our theoretical findings. The ROC curve of the proposed algorithm, Asymptotic Normal Thresholding (ANT), for support recovery significantly outperforms that of the popular GLasso algorithm.

Subfields: Applied Mathematics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Statistics and Probability

Citations in the corpus

The hardness of conditional independence testing and the generalised covariance measure (2018, methodology): “[41] and the socalled RP test introduced in Shah and Bühlmann [46], which amount to a regularised partial correlation.”
Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, background): “…2015); Javanmard and Montanari (2014a,b, 2018); Van de Geer et al. (2014); Ning and Liu (2017); Chernozhukov et al. (2015); Neykov et al. (2018); Ren et al. (2015); Jankova and Van De Geer (2015, 2016, 2018); Bradic and Kolar (2017); Zhu and Bradic (2017, 2018)], which focuses on coefficients…”
Semiparametric efficiency bounds for high-dimensional models (2016, methodology): “Thus, after normalization by √n and by the (estimated) standard deviation, it follows that it is asymptotically standard normal and minimax optimal [see Ren et al. (2015), Janková and van de Geer (2017)].”
Structure Learning in Graphical Modeling (2016, background): “Bayesian approaches naturally include an uncertainty assessment but frequentist techniques that are able to cope with high-dimensional data are also being developed (Jankov´a & van de Geer, 2015; Ren et al., 2015; Xia et al., 2015).”
Robust regression via mutivariate regression depth (2017, unknown): “This class was previously considered in [27].”
Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation (2016, methodology): “Ren et al. (2015) further removed the matrix 1 norm condition based on the estimator Ω̂ANT obtained in (25).”
A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations (2015, background): “Once again, the first part of condition (3.6) agrees with Ren et al. (2015); Liu (2013).”
Confidence intervals for causal effects with invalid instruments by using two‐stage hard thresholding with voting (2016, unknown): “Proof of (158) A stronger version of this proposition has already been proved in Ren et al. (2013), where part of it was already established in Sun and Zhang (2012).”
High-dimensional inference in misspecified linear models (2015, background): “nari, 2014; Meinshausen, 2015; Foygel Barber and Candes, 2014), but generalized linear models (Meinshausen et al., 2009; Minnier et al., 2011; van de Geer et al., 2014), undirected graphical models (Ren et al., 2015; Jankova and van de Geer, 2014), instrumental variable models (Belloni et al., 2012)”
Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures (2017, unknown): “Ren, Z., T. Sun, C.-H. Zhang, and H. H. Zhou (2015).”
Double/de-biased machine learning using regularized Riesz representers (2018, background): “…2015); Javanmard and Montanari (2014a,b, 2015); van de Geer et al. (2014); Ning and Liu (2014); Chernozhukov et al. (2015); Neykov et al. (2015); Ren et al. (2015); Jankova and Van De Geer (2015, 2016a,b); Bradic and Kolar (2017); Zhu and Bradic (2017b,a)], which focused on inference on the…”
Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity (2015, background): “[19] Zhao Ren, Tingni Sun, Cun-Hui Zhang, and Harrison H Zhou.”
Learning L2 Continuous Regression Functionals via Regularized Riesz Representers (2018, background): “ as in Zhang and Zhang (2014), Belloni Chernozhukov, and Wang (2014), Belloni, Chernozhukov, and Kato (2015), Javanmard and Montanari (2014a,b; 2015), van de Geer et al. (2014), Neykov et al. (2015), Ren et al. (2015), Jankova and van de Geer (2015, 2 2016a,b), Bradic and Kolar (2017); Zhu and Bradi”