Confidence intervals for low dimensional parameters in high dimensional linear models

By Cun-Hui Zhang, Shenmin Zhang. 2011

1,156 citations recorded by the source corpus.

The purpose of this paper is to propose methodologies for statistical inference of low dimensional parameters with high dimensional data. We focus on constructing confidence intervals for individual coefficients and linear combinations of several of them in a linear regression model, although our ideas are applicable in a much broader context. The theoretical results that are presented provide sufficient conditions for the asymptotic normality of the proposed estimators along with a consistent estimator for their finite dimensional covariance matrices. These sufficient conditions allow the number of variables to exceed the sample size and the presence of many small non‐zero coefficients. Our methods and theory apply to interval estimation of a preconceived regression coefficient or contrast as well as simultaneous interval estimation of many regression coefficients. Moreover, the method proposed turns the regression data into an approximate Gaussian sequence of point estimators of individual regression coefficients, which can be used to select variables after proper thresholding. The simulation results that are presented demonstrate the accuracy of the coverage probability of the confidence intervals proposed as well as other desirable properties, strongly supporting the theoretical results.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, positive): “Our paper also builds upon and contributes to the literature on modern orthogonal/debiased estimation and inference [Zhang and Zhang (2014); Belloni et al. (2011, 2014, 2015); Javanmard and Montanari (2014a,b, 2018); Van de Geer et al. (2014); Ning and Liu (2017); Chernozhukov et al. (2015); Neykov…”
Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, methodology): “In addition to approximate residual balancing as described in procedure 1, the methods that we use as baselines are as follows: naive difference-in-means estimation τ̂ = Ȳt− Ȳc that ignores the covariate information X; the elastic net (Zou and Hastie, 2005) or, equivalently, procedure 1 with trivial”
Distributed Multi-Task Learning (2015, methodology): “For example, the group lasso penalty uses pen( t ∈ m β 2 tj [Yuan and Lin, 2006], while the iCAP uses pen( Zhao et al., 2009].”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, methodology): “The estimator b̂Lasso corresponds to the proposal in [52].”
Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data (2018, methodology): “Our work is also connected to the literature of confidence intervals and hypothesis testing for a single or lower-dimensional coefficients in high-dimensional regression models (Zhang and Zhang (2014); van de Geer et al. (2014); Javanmard and Montanari (2014)).”
Confidence intervals for high-dimensional inverse covariance estimation (2014, methodology): “The paper [13] essentially follows the same approach as [39] but uses a different approach to find an approximate inverse for the sample covariance matrix.”
High-dimensional econometrics and regularized GMM (2018, background): “Important seminal contributions following the de-biasing approach are [124], [109], and [79].”
Controlling the false discovery rate via knockoffs (2014, background): “Finally, recent work [4, 15, 27, 28, 31] extends the classical notions of confidence intervals and p-values into the high-dimensional setting (p n); although the coefficients of the linear model are no longer identifiable in the classical sense,”
Asymptotic normality and optimalities in estimation of large Gaussian graphical models (2013, methodology): “A proof of the asymptotic efficiency of the LDPE was given in van de Geer et al. (2014) where the generalized linear model was also considered.”
Debiased Inference of Average Partial Effects in Single-Index Models (2018, methodology): “…There has recently been a considerable amount of interest in developing methods for statistical inference in high-dimensional regimes with more covariates than data points [Athey et al., 2018, Belloni et al., 2017, Javanmard and Montanari, 2014, van de Geer et al., 2014, Zhang and Zhang, 2014].”
A Projection-based Conditional Dependence Measure with Applications to High-dimensional Undirected Graphical Models. (2015, methodology): “It is worth mentioning that an alternative approach to reduce the estimation bias is the de-biased lasso method (Zhang and Zhang, 2014; Van de Geer et al., 2014).”
Confidence intervals for causal effects with invalid instruments by using two‐stage hard thresholding with voting (2016, neutral): “While some recent work in high dimensional inference (Zhang and Zhang, 2014; Javanmard and Montanari, 2014; van de Geer et al., 2014; Chernozhukov et al., 2015; Cai and Guo, 2017a) does not make this “well separated” assumption, our invalid IV problem is different from the prior work because a singl”
Efficient Distributed Learning with Sparsity (2016, methodology): “In particular, the debiasing step requires solving Oppq generalized lasso problems, which is computationally prohibitive for high-dimensional problems (Zhang and Zhang, 2013; Javanmard and Montanari, 2014).”