Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies

By Peter C. Austin. Pharmaceutical statistics, 2010

3,326 citations recorded by the source corpus.

In a study comparing the effects of two treatments, the propensity score is the probability of assignment to one treatment conditional on a subject's measured baseline covariates. Propensity‐score matching is increasingly being used to estimate the effects of exposures using observational data. In the most common implementation of propensity‐score matching, pairs of treated and untreated subjects are formed whose propensity scores differ by at most a pre‐specified amount (the caliper width). There has been a little research into the optimal caliper width. We conducted an extensive series of Monte Carlo simulations to determine the optimal caliper width for estimating differences in means (for continuous outcomes) and risk differences (for binary outcomes). When estimating differences in means or risk differences, we recommend that researchers match on the logit of the propensity score using calipers of width equal to 0.2 of the standard deviation of the logit of the propensity score. When at least some of the covariates were continuous, then either this value, or one close to it, minimized the mean square error of the resultant estimated treatment effect. It also eliminated at least 98% of the bias in the crude estimator, and it resulted in confidence intervals with approximately the correct coverage rates. Furthermore, the empirical type I error rate was approximately correct. When all of the covariates were binary, then the choice of caliper width had a much smaller impact on the performance of estimation of risk differences and differences in means. Copyright © 2010 John Wiley & Sons, Ltd.

Subfields: Economics and Econometrics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8c00c952667863cfea4761e0bf9b4944d94e2afd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies (2011, background): “Recently, Austin (2011b) examined optimal caliper widths when estimating risk differences and differences in means.”
Signaling by early stage startups: US government research grants and venture capital funding (2018, background): “Austin (2011) recommended this to be the minimum caliper width to minimize the mean squared error (MSE) when at least one of the covariates is continuous, as in our case.”
The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments (2013, background): “2 of the standard deviation of the logit of the propensity score, as this caliper has been shown to be optimal in a range of settings [60].”
Estimation of causal effects with multiple treatments: a review and new ideas (2017, methodology): “WithinTrio uses the KD-tree algorithm (Moore, 1991) to optimize triplet similarities based on units’ GPS’s for treatments t1 and treatments t2, by using a distance function between all possible pairs of triplets (Hott, Brunelle and Myers, 2012). Using simulations, Rassen et al. (2013) found that tri”
Shining a spotlight on small rural businesses: How does their performance compare with urban? (2019, methodology): “Its application spans a diverse set of policy fields from care for patients with cardiovascular disease (Austin, 2011) to the effectiveness of on-the-job training (Dehejia and Wahba, 2002).”
Employment Protection, Investment, and Firm Growth (2019, methodology): “Electronic copy available at: https://ssrn.com/abstract=2705066 28 replacement such that the logit of the treatment and control firms’ propensity scores are within 0.2 standard deviations of each other (Austin, 2011).23 Fifth, we create two samples of control firms.”
The performance of different propensity score methods for estimating marginal hazard ratios (2007, background): “2 of the standard deviation of the logit of the propensity score as this caliper width has been found to perform well in a wide variety of settings [14].”
A comparison of 12 algorithms for matching on the propensity score (2013, methodology): “For the nearest neighbor matching algorithms, we matched subjects on the propensity score, whereas in the caliper matching algorithms, we matched subjects on the logit of the propensity score using a caliper of width equal to 0:2 of the standard deviation of logit of the propensity score [11].”
Association Between Wait Time and 30-Day Mortality in Adults Undergoing Hip Fracture Surgery (2017, methodology): “To further evaluate the robustness of this definition, early and delayed patients werematched 1:1 without replacement(28) on the logit of propensity scores.(29) Restricted cubic spline, logistic regression, and propensity scoremodelswereall adjusted for thesamecovariates: age, sex, year, income quin”