When Should You Adjust Standard Errors for Clustering?

By Alberto Abadie, Susan Athey, Guido W. Imbens, Jeffrey M. Wooldridge. Quarterly Journal of Economics, 2017

2,001 citations recorded by the source corpus.

Clustered standard errors, with clusters defined by factors such as geography, are widespread in empirical research in economics and many other disciplines. Formally, clustered standard errors adjust for the correlations induced by sampling the outcome variable from a data-generating process with unobserved cluster-level components. However, the standard econometric framework for clustering leaves important questions unanswered: (i) Why do we adjust standard errors for clustering in some ways but not others, e.g., by state but not by gender, and in observational studies, but not in completely randomized experiments? (ii) Why is conventional clustering an “all-or-nothing” adjustment, while within-cluster correlations can be strong or extremely weak? (iii) In what settings does the choice of whether and how to cluster make a difference? We address these and other questions using a novel framework for clustered inference on average treatment effects. In addition to the common sampling component, the new framework incorporates a design component that accounts for the variability induced on the estimator by the treatment assignment mechanism. We show that, when the number of clusters in the sample is a nonnegligible fraction of the number of clusters in the population, conventional cluster standard errors can be severely inflated, and propose new variance estimators that correct for this bias.

Subfields: Economics and Econometrics, Marketing

View Paper: https://www.semanticscholar.org/paper/b1a0d2f7b570546460d2d69af5421b3b41e44304

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Marketing

Citations in the corpus

What’s trending in difference-in-differences? A synthesis of the recent econometrics literature (2022, positive): “Abadie et al. (2023) study a model in which both treatment is stochastic and units are sampled from a larger population, and suggest that one should cluster among units if either their treatment assignments are correlated or the event that they are included in the sample is correlated.”
Panel experiments and dynamic causal effects: A finite population perspective (2020, background): “, Abadie et al. (2017), Athey and Imbens (2018), Athey et al. (2018), Imai and Kim (2019), Arkhangelsky and Imbens (2019), Imai and Kim (2020), de Chaisemartin and D’Haultfoeuille (2020)).”
Cluster-robust inference: A guide to empirical practice (2022, methodology): “The “design-based” approach to inference, which is analyzed in detail in Abadie et al. (2017), is conceptually different from the model-based framework.”
Revisiting regression adjustment in experiments with heterogeneous treatment effects (2020, methodology): “For a deeper discussion of sampling-based and designbased uncertainty, see Abadie et al. (2017b). Our primary motivation for studying this problem comes from Imbens and Rubin (2015), who study linear regression adjustment in the same sampling setting that we use here: independent and identically dis”
Contamination Bias in Linear Regressions (2021, positive): “We use robust (non-clustered) standard errors throughout this analysis, since the randomization of students to classrooms is at the individual level (Abadie et al., 2017).”
Difference-in-Differences with multiple time periods (2018, background): “On a different direction, it is also interesting to extend our inference procedures to incorporate not only the sampling uncertainty as we do in this article, but also the design uncertainty as in Abadie et al. (2017) and Athey and Imbens (2018).”
Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R (2020, background): “Thus, in the terminology of Abadie et al. (2017), the parameter ρ above controls the correlation in the sampling process while the parameter ρ x controls the extent of the clustering in the “treatment” assignment.”
Minimum Wages and Racial Inequality* (2020, background): “We view clustering here mainly as an experimental design issue where the assignment is correlated within the clusters (see Abadie et al. 2017).”
Robots and reshoring: Evidence from Mexican labor markets (2020, background): “Abadie et al. (2017) show that in models without fixed effects, a necessary condition for clustering standard errors is clustering in the assignment of a treatment.”