Sensitivity of Propensity Score Methods to the Specifications

By Zhong Zhao. Social Science Research Network, 2008

158 citations recorded by the source corpus.

Propensity score matching estimators have two advantages. One is that they overcome the curse of dimensionality of covariate matching, and the other is that they are nonparametric. However, the propensity score is usually unknown and needs to be estimated. If we estimate it nonparametrically, we are incurring the curse-of-dimensionality problem we are trying to avoid. If we estimate it parametrically, how sensitive the estimated treatment effects are to the specifications of the propensity score becomes an important question. In this paper, we study this issue. First, we use a Monte Carlo experimental method to investigate the sensitivity issue under the unconfoundedness assumption. We find that the estimates are not sensitive to the specifications. Next, we provide some theoretical justifications, using the insight from Rosenbaum and Rubin (1983) that any score finer than the propensity score is a balancing score. Then, we reconcile our finding with the finding in Smith and Todd (2005) that, if the unconfoundedness assumption fails, the matching results can be sensitive. However, failure of the unconfoundedness assumption will not necessarily result in sensitive estimates. Matching estimators can be speciously robust in the sense that the treatment effects are consistently overestimated or underestimated. Sensitivity checks applied in empirical studies are helpful in eliminating sensitive cases, but in general, it cannot help to solve the fundamental problem that the matching assumptions are inherently untestable. Last, our results suggest that including irrelevant variables in the propensity score will not bias the results, but overspecifying it (e.g., adding unnecessary nonlinear terms) probably will.

View Paper: https://www.semanticscholar.org/paper/e87bdff1be464e6cecb6fa62b1b8db562ac1fcec

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Commuting Time and Household Responsibilities: Evidence Using Propensity Score Matching (2016, unknown): “…matching over regression analysis is that it is non-parametric: matching does not impose functional form restrictions, such as linearity and homogeneous effects on the distribution of covariates, both assumptions being usually unjustified, either by the economic theory or by the data (Zhao, 2008).”
Propensity score matching and variations on the balancing test (2011, unknown): “Using a propensity score model comprised of two covariates from artificially generated data, Zhao (2008) finds that misspecification of the propensity score model is relatively unimportant when the CIA holds in the sense that the poorly estimated coefficients have little influence on the estimated…”
Why Propensity Scores Should Not Be Used for Matching (2019, background): “For example, estimating the propensity score regression with misspecification can bias estimates (Drake, 1993; Smith and Todd, 2005; Kang and Schafer, 2007; Zhao, 2008; Diamond and Sekhon, 2012).”
Which covariates should be controlled in propensity score matching? Evidence from a simulation study (2013, methodology): “This design is population in simulation studies of impact evaluation (e.g. see Augurzky and Schmidt, 2001; Zhao, 2004, 2008).”
The performance of estimators based on the propensity score (2013, background): “…as it remains consistent if the misspecified propensity score model is a monotone transformation of the true model (see the simulation results in Drake, 1993, Zhao, 2004, 2008, Millimet and Tchernis, 2009, and Huber, 2011, suggesting some robustness to the specification of the propensity score).”