The Econometrics of Randomized Experiments

By Susan Athey, Guido W. Imbens. 2016

465 citations recorded by the source corpus.

Abstract In this chapter, we present econometric and statistical methods for analyzing randomized experiments. For basic experiments, we stress randomization-based inference as opposed to sampling-based inference. In randomization-based inference, uncertainty in estimates arises naturally from the random assignment of the treatments, rather than from hypothesized sampling from a large population. We show how this perspective relates to regression analyses for randomized experiments. We discuss the analyses of stratified, paired, and clustered randomized experiments, and we stress the general efficiency gains from stratification. We also discuss complications in randomized experiments such as noncompliance. In the presence of noncompliance, we contrast intention-to-treat analyses with instrumental variables analyses allowing for general treatment effect heterogeneity. We consider, in detail, estimation and inference for heterogenous treatment effects in settings with (possibly many) covariates. These methods allow researchers to explore heterogeneity by identifying subpopulations with different treatment effects while maintaining the ability to construct valid confidence intervals. We also discuss optimal assignment to treatment based on covariates in such settings. Finally, we discuss estimation and inference in experiments in settings with interactions between units, both in general network settings and in settings where the population is partitioned into groups with all interactions contained within these groups.

Subfields: Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Machine Learning Methods That Economists Should Know About (2019, methodology): “The panel data literature traditionally focused on fixed effect methods, and has generalized those to models with multiple latent factors (Bai and Ng [2017, 2002], Bai [2003]) that are essentially the same as the low rank factorizations in the ML literature.”
The Generalized Oaxaca-Blinder Estimator (2020, background): “…inference has seen a revival in causal inference, both in terms of theoretical developments (Bloniarz et al. 2016; Li and Ding 2017; Mukerjee, Dasgupta, and Rubin 2018; Li and Ding 2020) and in terms of its popularity with applied researchers (Hansen and Bowers 2009; Athey and Imbens 2017).”
Identification and Estimation of Spillover Effects in Randomized Experiments (2017, background): “A thorough account of spillover effects is crucial to assess the causal impact of policies and programs (Abadie and Cattaneo, 2018; Athey and Imbens, 2017).”
Closing the Gap: The Effect of a Targeted, Tuition-Free Promise on College Choices of High-Achieving, Low-Income Students (2018, methodology): “We demonstrate this with a simulation exercise following Athey and Imbens (2017), who recommend randomization-based statistical inference for significance tests.”
High-dimensional regression adjustments in randomized experiments (2016, background): “…the number of samples grows to inﬁnity, it is now well-established that regression adjustments are always asymptotically helpful—even in misspeciﬁed models—provided we add full treatment-by-covariate interactions to the regression design and use robust standard errors [2, 3, 4, 5, 6, 7, 8, 9, 10].”
Rerandomization and regression adjustment (2019, methodology): “This remains a powerful tool in modern experiments (Miratrix et al., 2013; Higgins et al., 2016; Athey and Imbens, 2017).”
When Should You Adjust Standard Errors for Clustering? (2017, unknown): “[2009], Conley [1999], Barrios et al. [2012], Cressie [2015]).”
Optimality of Matched-Pair Designs in Randomized Controlled Trials (2022, methodology): “I also consider several stratifications with matched sets of four, as in Athey and Imbens (2017). Specifically, strata are constructed by first pairing the units and then pairing the pairs according to their midpoints, as in Section 4 of Bai et al.”
Econometric Methods for Program Evaluation (2018, background): “Athey and Imbens (2017a) provide a detailed introduction to the econometrics of randomized experiments.”
Rethinking the Benefits of Youth Employment Programs: The Heterogeneous Effects of Summer Jobs (2017, unknown): “This tests the sharp null of no treatment effects for anyone and avoids relying on modeling assumptions and large-sample approximations that may not hold in finite samples (Athey & Imbens, 2017).”