Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples

By Peter C. Austin. Statistics in Medicine, 2009

5,938 citations recorded by the source corpus.

The propensity score is a subject's probability of treatment, conditional on observed baseline covariates. Conditional on the true propensity score, treated and untreated subjects have similar distributions of observed baseline covariates. Propensity‐score matching is a popular method of using the propensity score in the medical literature. Using this approach, matched sets of treated and untreated subjects with similar values of the propensity score are formed. Inferences about treatment effect made using propensity‐score matching are valid only if, in the matched sample, treated and untreated subjects have similar distributions of measured baseline covariates. In this paper we discuss the following methods for assessing whether the propensity score model has been correctly specified: comparing means and prevalences of baseline characteristics using standardized differences; ratios comparing the variance of continuous covariates between treated and untreated subjects; comparison of higher order moments and interactions; five‐number summaries; and graphical methods such as quantile–quantile plots, side‐by‐side boxplots, and non‐parametric density plots for comparing the distribution of baseline covariates between treatment groups. We describe methods to determine the sampling distribution of the standardized difference when the true standardized difference is equal to zero, thereby allowing one to determine the range of standardized differences that are plausible with the propensity score model having been correctly specified. We highlight the limitations of some previously used methods for assessing the adequacy of the specification of the propensity‐score model. In particular, methods based on comparing the distribution of the estimated propensity score between treated and untreated subjects are uninformative. Copyright © 2009 John Wiley & Sons, Ltd.

View Paper: https://www.semanticscholar.org/paper/acb971dad4887887b13c73e24c29006d69ace08a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies (2015, methodology): “Similarly, in the context of propensity score matching, comparing the empirical distribution of the propensity score does not serve as an indication of whether the propensity score model has been correctly specified [10].”
A comparison of 12 algorithms for matching on the propensity score (2013, methodology): “We also computed the absolute standardized difference comparing the distribution of each of the 10 baseline covariates between treatment groups in each of the matched samples [18, 19].”
Balance diagnostics after propensity score matching. (2019, methodology): “Similarly, SMD for dichotomous variable is given by (6):”
Referral to an extracorporeal membrane oxygenation center and mortality among patients with severe 2009 influenza A(H1N1). (2011, methodology): “The balance statistics are chosen a priori from recommended measures such as t statistics from paired t tests, D statistics from Kolmogorov-Smirnov tests, and weighted standardized differences.(25) The search algorithm improves covariate balance to the extent possible given the data.”
Association Between Wait Time and 30-Day Mortality in Adults Undergoing Hip Fracture Surgery (2017, methodology): “1 were considered indicative of imbalance.(31) The McNemar test compared outcomes in early and delayed groups after matching and generalized estimating equations (GEEs) calculated percent absolute risk differences (% absolute risk difference [RD], with 95% CIs).”
An introduction to inverse probability of treatment weighting in observational research (2021, methodology): “As IPTW aims to balance patient characteristics in the exposed and unexposed groups, it is considered good practice to assess the standardized differences between groups for all baseline characteristics both before and after weighting [22].”
A unified approach to measuring the effect size between two groups using SAS (2012, unknown): “Austin (2009) proposed methods to “estimate the empirical sampling distribution of the standardized difference of the mean under the assumption that the mean (or prevalence) of a covariate is equal between two groups” [13].”
Estimation of causal effects with multiple treatments: a review and new ideas (2017, methodology): “WithinTrio uses the KD-tree algorithm (Moore, 1991) to optimize triplet similarities based on units’ GPS’s for treatments t1 and treatments t2, by using a distance function between all possible pairs of triplets (Hott, Brunelle and Myers, 2012). Using simulations, Rassen et al. (2013) found that tri”
An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies (2011, methodology): “The use of IPTW is similar to the use of survey sampling weights that are used to weight survey samples so that they are representative of specific populations (Morgan & Todd, 2008).”
The role of prediction modeling in propensity score estimation: an evaluation of logistic regression, bCART, and the covariate-balancing propensity score. (2014, methodology): “The potential disconnect between the prediction of treatment assignment and obtaining covariate balance has led to the recommendation and practice of using measures of covariate balance to evaluate estimated PS models (9, 10).”
Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies (2010, unknown): “propensity-score matching is provided elsewhere [28], as is a comparison of the relative ability of the different propensityscore methods to balance measured covariates [29].”
The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments (2013, methodology): “While balance diagnostics have been described for covariate adjustment using the propensity score, these diagnostics are less transparent than comparable diagnostics for the other three approaches [48, 49].”