A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models

By Yang Ning, Han Liu. 2014

350 citations recorded by the source corpus.

We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. Unlike most existing inferential methods which are tailored for individual models, our approach provides a general framework for high dimensional inference and is applicable to a wide range of applications. From the testing perspective, we develop general theorems to characterize the limiting distributions of the decorrelated score test statistic under both null hypothesis and local alternatives. These results provide asymptotic guarantees on the type I errors and local powers of the proposed test. Furthermore, we show that the decorrelated score function can be used to construct point and confidence region estimators that are semiparametrically efficient. We also generalize this framework to broaden its applications. First, we extend it to handle high dimensional null hypothesis, where the number of parameters of interest can increase exponentially fast with the sample size. Second, we establish the theory for model misspecification. Third, we go beyond the likelihood framework, by introducing the generalized score test based on general loss functions. Thorough numerical studies are conducted to back up the developed theoretical results.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, positive): “…and inference [Zhang and Zhang (2014); Belloni et al. (2011, 2014, 2015); Javanmard and Montanari (2014a,b, 2018); Van de Geer et al. (2014); Ning and Liu (2017); Chernozhukov et al. (2015); Neykov et al. (2018); Ren et al. (2015); Jankova and Van De Geer (2015, 2016, 2018); Bradic and…”
Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, positive): “…effects without needing to estimate treatment propensities as nuisance parameters; rather, we build on recent developments on inference in high dimensional linear models (Cai and Guo, 2017; Javanmard and Montanari, 2014, 2015; Ning and Liu, 2017; Van de Geer et al., 2014; Zhang and Zhang, 2014).”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, methodology): “(2014), Belloni, Chernozhukov, and Hansen (2014), Belloni, Chernozhukov, and Kato (2015), and Ning and Liu (2017). Recall the example in Remark 3.”
Statistical inference for model parameters in stochastic gradient descent (2016, background): “Robbins and Monro (1951); Polyak and Juditsky (1992); Nemirovski et al.”
Learning L2 Continuous Regression Functionals via Regularized Riesz Representers (2018, background): “Unlike Ning and Liu (2017), we do not work with an explicit semiparametric form for the distribution of the data.”
A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations (2015, methodology): “We emphasize that the construction of v̂ does not depend on knowing which is the estimating equation for θ and which is the nuisance estimating equation space, and thus the projection is different from the decorrelated score method in Ning and Liu (2014a).”
Beyond Gaussian approximation: Bootstrap for maxima of sums of independent random vectors (2017, background): “These examples cover many practical problems and applications as discussed in Chernozhukov et al. (2013, 2017), and many publications citing their work (Dezeure et al., 2017; Ning and Liu, 2017; Zhang and Wu, 2017b; Chen et al., 2018; Blanchet et al., 2019; Horowitz, 2019).”
Breaking the curse of dimensionality in regression (2017, neutral): “To name a few Van de Geer et al. (2014), Zhang and Zhang (2014), Javanmard and Montanari (2014a) (JM), Ning and Liu (2014) (NL), Belloni et al. (2014) allow the sparsity to grow at the rate o( √ n/ log p) and design tests for testing univariate parameters in high-dimensional sparse models.”
Double/de-biased machine learning using regularized Riesz representers (2018, background): “…and inference [Zhang and Zhang (2014); Belloni et al. (2014a,b, 2015); Javanmard and Montanari (2014a,b, 2015); van de Geer et al. (2014); Ning and Liu (2014); Chernozhukov et al. (2015); Neykov et al. (2015); Ren et al. (2015); Jankova and Van De Geer (2015, 2016a,b); Bradic and Kolar…”
High-dimensional econometrics and regularized GMM (2018, background): “Other uses of C(α)-statistics or close variants include those in [114], [99], [119], and [98].”
A modern maximum-likelihood theory for high-dimensional logistic regression (2018, background): “Another approach is to extend the score test to the ultra high-dimensional setting as done in [77]; here, the authors consider five different models that includes logistic regression in particular.”
Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score. (2018, background): “…high-dimensional inference on regression models; see Zhang & Zhang (2014), Javanmard &70 Montanari (2014), Van de Geer et al. (2014), Belloni et al. (2016), Ning & Liu (2017); Ning et al. (2017), Cai & Guo (2017), Neykov et al. (2018) Zhu & Bradic (2018), and Dukes et al. (2018), among many others.”