Models as Approximations I: Consequences Illustrated with Linear Regression

By Andreas Buja, Richard A. Berk, Lawrence D. Brown, Edward I. George, E. Pitkin, Mikhail Traskin, Linda H. Zhao, Kai Zhang. Statistical Science, 2014

101 citations recorded by the source corpus.

In the early 1980s Halbert White inaugurated a "model-robust'' form of statistical inference based on the "sandwich estimator'' of standard error. This estimator is known to be "heteroskedasticity-consistent", but it is less well-known to be "nonlinearity-consistent'' as well. Nonlinearity, however, raises fundamental issues because in its presence regressors are not ancillary, hence can't be treated as fixed. The consequences are deep: (1)~population slopes need to be re-interpreted as statistical functionals obtained from OLS fits to largely arbitrary joint $\xy$~distributions; (2)~the meaning of slope parameters needs to be rethought; (3)~the regressor distribution affects the slope parameters; (4)~randomness of the regressors becomes a source of sampling variability in slope estimates; (5)~inference needs to be based on model-robust standard errors, including sandwich estimators or the $\xy$~bootstrap. In theory, model-robust and model-trusting standard errors can deviate by arbitrary magnitudes either way. In practice, significant deviations between them can be detected with a diagnostic test.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b0188a56a913ffeaf2ead5d85dd55c1a7d0f8844

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Semiparametric Counterfactual Density Estimation (2021, methodology): “…where a model is not assumed correct but instead only used for defining approximations, has been used widely throughout statistics [Beran, 1977, Buja et al., 2019a,b, Huber, 1967, Rakhlin et al., 2017, Rinaldo and Wasserman, 2010, Tsybakov, 2003, Wasserman, 2006, White, 1982, 1996] as well as…”
A Hierarchy of Limitations in Machine Learning (2020, background): “…1998; Freedman, 2009), interpreting probability (Freedman, 2009; Cox, 1990), and determining what the “inferences” of statistical inference are to (Kass, 2011), especially given model misspecification (Buja et al., 2014); so long as out-of-sample success can be demonstrated, none of that matters.”
Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression (2018, background): “ar regression slope estimator for regressing Y on X. Under a possibly misspecied linear model, the estimation of the asymptotic variance of ^ involves P X2 i (Y i X i)2, where is the limit of ^; seeBuja et al. (2019) for details. It is clear that if the initial random variables Xand Y are sub-expone”