Post-Selection Inference for Generalized Linear Models With Many Controls

By Alexandre Belloni, Victor Chernozhukov, Ying Wei. 2013

206 citations recorded by the source corpus.

This article considers generalized linear models in the presence of many controls. We lay out a general methodology to estimate an effect of interest based on the construction of an instrument that immunizes against model selection mistakes and apply it to the case of logistic binary choice model. More specifically we propose new methods for estimating and constructing confidence regions for a regression parameter of primary interest α0, a parameter in front of the regressor of interest, such as the treatment variable or a policy variable. These methods allow to estimate α0 at the root-n rate when the total number p of other regressors, called controls, potentially exceeds the sample size n using sparsity assumptions. The sparsity assumption means that there is a subset of s < n controls, which suffices to accurately approximate the nuisance part of the regression function. Importantly, the estimators and these resulting confidence regions are valid uniformly over s-sparse models satisfying s2log 2p = o(n) and other technical conditions. These procedures do not rely on traditional consistent model selection arguments for their validity. In fact, they are robust with respect to moderate model selection mistakes in variable selection. Under suitable conditions, the estimators are semi-parametrically efficient in the sense of attaining the semi-parametric efficiency bounds for the class of models in this article.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/23c1020d04220c7c5277c9916ea6ce411e06f2d8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

High-dimensional econometrics and regularized GMM (2018, methodology): “The use of C(α) statistics for testing and estimation with high-dimensional approximately sparse models was first explored in the context of quantile regression in [19] and in the context of high-dimensional generalized linear models by [22].”
Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models (2013, methodology): “Although related in spirit with our previous work (Belloni, Chernozhukov, and Wei 2013a; Belloni, Chernozhukov, and Hansen 2014; Belloni, Chernozhukov, andKato 2014), new tools and major departures from the previous works are required.”
A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations (2015, methodology): “In particular, Belloni et al. (2012, 2013); Zhang and Zhang (2014); Javanmard and Montanari (2013); van de Geer et al. (2014) proposed a double selection estimator, low dimension projection estimator, debiasing and desparsifying correction methods, respectively, for constructing confidence…”
Double/Debiased Machine Learning for Treatment and Causal Parameters (2016, background): “ to hold while allowing moderate growth of the complexity/entropy of G N. Examples of papers taking this approach in an approximately sparse settings are Belloni et al. (2017), Belloni et al. (2014), Belloni et al. (2016), Chernozhukov et al. (2015b), Javanmard and Montanari (2014a), van de Geer et ”
Orthogonal Machine Learning for Demand Estimation: High Dimensional Causal Inference in Dynamic Panels (2017, methodology): “We impose weak sparsity assumption: (see, e.g. 7Partialling out unobserved heterogeneity in a high-dimensional sparse model was considered in [Belloni et al., 2016a].”
Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations (2013, unknown): “In independent work, Belloni, Chernozhukov, and Wei (2013) study exactly sparse logistic regression, also using Bach’s (2010) tools, but are focused on a different inference goal.”
Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions (2017, methodology): “…machine learning methods to estimate the nuisance function η , such as l 1 penalized methods in sparse models (B¨uhlmann and van der Geer (2011), Belloni et al. (2016)), L 2 boosting in sparse linear models (Luo and Spindler (2016)), and other methods for classes of neural nets, regression…”
Double/debiased machine learning for treatment and structural parameters (2017, unknown): “In the partially linear model, the DML approach complements Belloni et al. (2013), Zhang and Zhang (2014), van de Geer et al. (2014), Javanmard and Montanari (2014b), and Belloni et al. (2014), Belloni et al. (2014), and Belloni et al. (2015), all of which consider estimation and inference for…”
Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score. (2018, background): “…works on high-dimensional inference on regression models; see Zhang & Zhang (2014), Javanmard &70 Montanari (2014), Van de Geer et al. (2014), Belloni et al. (2016), Ning & Liu (2017); Ning et al. (2017), Cai & Guo (2017), Neykov et al. (2018) Zhu & Bradic (2018), and Dukes et al. (2018),…”
Predicting the long-term stock market volatility: A GARCH-MIDAS model with variable selection (2020, background): “Equity market returns (MKT) in Fama and French (1992) can capture the leverage effect (Black, 1976; Glosten et al., 1993; Christiansen et al., 2012). Nagel (2012) finds that the short-term reversal factor (STR) can be related to market volatility.”
Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores (2017, methodology): “The latter can be treated as a condition on the initial step estimator – see Belloni et al. (2013) for more details where the authors introduce an additional step to remove this assumption.”
Estimation and Inference about Heterogeneous Treatment Effects in High-Dimensional Dynamic Panels. (2017, methodology): “setup of Example 2. Let the sales and price reduced form be li0(Zit) = l0(Zit) +ξi pi0(Zit) = p0(Zit)+ηi 6Partialling out unobserved heterogeneity in a high-dimensional sparse model was considered in Belloni et al. (2016a). 22 where the controls Zit ≡ [Y′ i,t−1,Y ′ i,t−2,...,Y ′ i,t−L,P ′ i,t−1,P ′ ”