Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands

By Max H. Farrell, Tengyuan Liang, Sanjog Misra. 2018

269 citations recorded by the source corpus.

We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second‐step inference after first‐step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric inference, treat the current standard architecture: fully connected feedforward neural networks (multilayer perceptrons), with the now‐common rectified linear unit activation function, unbounded weights, and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed‐width, very deep networks. We establish the nonasymptotic bounds for these deep nets for a general class of nonparametric regression‐type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, and demonstrate the effectiveness of deep learning with an empirical application to direct mail marketing.

Subfields: Artificial Intelligence, Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Nonparametric estimation of causal heterogeneity under high-dimensional confounding (2019, result): “Similar rates can be shown for L2 boosting (Luo and Spindler, 2016) and nonlinear models like Random Forests (Wager and Walther, 2016) or forms of Deep Neural Nets (Farrell, Liang, and Misra, 2018).7 A natural question is then if a bandwidth exists that satisfies the rate conditions in Assumption 8.”
Machine Learning Advances for Time Series Forecasting (2020, methodology): “For Deep Neural Networks, Farrell et al. (2021) consider semiparametric inference and establish nonasymptotic high probability bounds.”
Semiparametric doubly robust targeted double machine learning: a review (2022, background): “Similar re-sults can be obtained for random forests, neural networks, etc., under appropriate conditions [Farrell et al., 2021].”
Estimation of Conditional Average Treatment Effects With High-Dimensional Data (2019, methodology): “For example, the L2 bounds for the random forest method and deep neural networks have already been established in Wager and Athey (2018) and Farrell, Liang, and Misra (2018), respectively.”
Policy Learning With Observational Data (2017, background): “19 In a recent advance, Farrell, Liang, and Misra (2018) established conditions under which deep neural known to satisfy Assumption 2 asymptotically and heuristic statistical learning tools, and then synthesize the output of all models via cross-validation.”
Covariate-adjusted Fisher randomization tests for the average treatment effect (2020, background): “2019; Guo and Basse 2020), and even estimators based on machine learning algorithms (Wager et al. 2016; Wu and Gagnon-Bartsch 2018; Farrell et al. 2020; Chen et al. 2020).”
Adapting Neural Networks for the Estimation of Treatment Effects (2019, methodology): “Farrell et al. [ Far + 18 ] prove that neural networks converge at a fast enough rate to invoke the double machine learning results.”