On the Failure of the Bootstrap for Matching Estimators

By Alberto Abadie, Guido W. Imbens. 2006

1,043 citations recorded by the source corpus.

Matching estimators are widely used in empirical economics for the evaluation of programs or treatments. Researchers using matching methods often apply the bootstrap to calculate the standard errors. However, no formal justification has been provided for the use of the bootstrap in this setting. In this article, we show that the standard bootstrap is, in general, not valid for matching estimators, even in the simple case with a single continuous covariate where the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Valid inferential methods in this setting are the analytic asymptotic variance estimator of Abadie and Imbens (2006a) as well as certain modifications of the standard bootstrap, like the subsampling methods in Politis and Romano (1994).

Subfields: Economics and Econometrics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/839451790ab91c36043c48bbe41e61061a175079

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

Matching and semi-parametric IV estimation, a distance-based measure of migration, and the wages of young men (2011, methodology): “Moreover, Abadie and Imbens (2008) show that the bootstrap is, in general, not valid for nearest neighbor matching, even when the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias.”
Do Payments for Environmental Services Affect Forest Cover? A Farm-Level Evaluation from Costa Rica (2012, methodology): “We use a postmatching biascorrection procedure that asymptotically removes the conditional bias in finite samples (Abadie and Imbens 2006b).”
Additionality and the Adoption of Farm Conservation Practices (2013, background): “However, Abadie and Imbens (2006, 2008) explain that bootstrapped standard errors are not valid for nearest-neighbor matching with a fixed number of neighbors, and further explain that kernel-based matching for which the number of matches increases with sample size has estimators that are…”
The effect of aid on growth: evidence from a Quasi-experiment (2014, neutral): “We did not do this because bootstrap fails for the K-nearest neighbor matching (Abadie and Imbens 2008).”
Kernel-Based Semiparametric Estimators: Small Bandwidth Asymptotics and Bootstrap Consistency (2018, result): “The approach we take is also similar to the approach taken by Abadie and Imbens (2006, 2008), but our main conclusion regarding the bootstrap (and subsampling) is quite different from that of Abadie and Imbens (2008).”
Are Training Programs More Effective When Unemployment Is High? (2009, methodology): “Moreover, this procedure should reduce small sample bias as well as asymptotic bias of matching estimators (see Abadie and Imbens, 2006) and thus increase robustness of the estimator.”
The Long-Run Dynamics of Electricity Demand: Evidence from Municipal Aggregation (2020, methodology): “D Inference Because matching estimators do not meet the regularity conditions required for bootstrapping (Abadie and Imbens, 2008), we employ a subsampling procedure to construct confidence intervals for our matching estimates.”
Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates (2012, background): “27 Abadie and Imbens (2008) demonstrate that, with nearest neighbor matching and a fixed number of matches per treatment unit, bootstrapping does not yield valid statistical inference for PSM.”
Bootstrap Inference of Matching Estimators for Average Treatment Effects (2017, methodology): “From Polya’s theorem and Pr{ √ N σN (τ̃ − τ ) ≤ r} → (r) for all r ∈ R (Abadie and Imbens 2006, Theorem 4), it is enough for (B.2) to verify Pr {√ N N∑ i=1 (W∗i − W̄∗)ηi σN ≤ r ∣∣∣∣∣Z } − (r) p→ 0 for all r ∈ R, (B.3) where (r) is the standard normal distribution function.”
Rural Roads and Local Market Development in Vietnam (2011, methodology): “This technique ensures valid bootstrapped standard errors (Abadie and Imbens, 2006).16 As a robustness check, we also construct a PS-weighted DD (Hirano and Imbens, 2002; Hirano et al., 2003).17 The key assumption of PS-matched or weighted DD in this context is that the selection bias is…”
Rural Roads and Poor Area Development in Vietnam (2007, methodology): “This technique ensures valid bootstrapped standard errors (Abadie and Imbens, 2006).”
Effect of Migration on Children's Educational Performance in Rural China (2007, methodology): “Although the above matching methods can significantly improve the reliability of matching estimators, producing results that have been shown to be very close to those based on a randomized design (Smith and Todd, 2005; Abadie and Imbens, 2006). Smith and Todd (2005)”