Improving propensity score weighting using machine learning

By Brian K. Lee, Justin Lessler, Elizabeth A. Stuart. Statistics in Medicine, 2010

924 citations recorded by the source corpus.

Machine learning techniques such as classification and regression trees (CART) have been suggested as promising alternatives to logistic regression for the estimation of propensity scores. The authors examined the performance of various CART‐based propensity score models using simulated data. Hypothetical studies of varying sample sizes (n=500, 1000, 2000) with a binary exposure, continuous outcome, and 10 covariates were simulated under seven scenarios differing by degree of non‐linear and non‐additive associations between covariates and the exposure. Propensity score weights were estimated using logistic regression (all main effects), CART, pruned CART, and the ensemble methods of bagged CART, random forests, and boosted CART. Performance metrics included covariate balance, standard error, per cent absolute bias, and 95 per cent confidence interval (CI) coverage. All methods displayed generally acceptable performance under conditions of either non‐linearity or non‐additivity alone. However, under conditions of both moderate non‐additivity and moderate non‐linearity, logistic regression had subpar performance, whereas ensemble methods provided substantially better bias reduction and more consistent 95 per cent CI coverage. The results suggest that ensemble methods, especially boosted CART, may be useful for propensity score weighting. Copyright © 2009 John Wiley & Sons, Ltd.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/0ecf8c56300c20622f317e1e6cefdeeb85c513e2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Balance diagnostics after propensity score matching. (2019, background): “Simulated study demonstrated that these advanced methods can help to improve balance between treated and untreated groups (16).”
Propensity score estimation: machine learning and classification methods as alternatives to logistic regression (2010, background): “[46] begin this work, further simulation studies are needed to compare the performances of these different techniques in different settings and with data structures with a wide range of strengths of exposureeoutcome, covariateeexposure, and covariateeoutcome relationships; covariance structures amon”
The Blessings of Multiple Causes (2018, background): “Schneeweiss et al. (2009), McCaffrey, Ridgeway, and Morral (2004), Lee, Lessler, and Stuart (2010), and many others develop and evaluate different models for assigned causes.”
Matching on Balanced Nonlinear Representations for Treatment Effects Estimation (2017, background): “Causal questions exist in many areas, such as health care [24, 12], economics [14], political science [17], education [36], digital marketing [6, 43, 5, 15, 44], etc.”
Supervised machine learning: A brief primer (2020, methodology): “For example, classification and regression trees and random forests have been used to generate propensity score models used to adjust treatment effect estimates based on individual differences in the (non-random) likelihood of receiving a specific intervention (Lee et al., 2010).”
Covariate balancing propensity score by tailored loss functions (2016, methodology): “ thanks Trevor Hastie, Hera Y. He and Dylan Small for valuable comments. 1 arXiv:1601.05890v2 [stat.ME] 17 Apr 2016 2 TAILORED LOSS FUNCTIONS and machine learning methods (e.g. McCarey et al., 2004, Lee et al., 2010, Wager and Athey, 2015) have also been suggested in the literature. Step 2: Adjust f”
Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores (2018, methodology): “Although some robust nonparametric methods (Hirano et al., 2003; Lee et al., 2010, 2011) can be used for propensity score estimation, the majority of the literature uses parametric generalized linear models.”
An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies (2011, methodology): “…McCaffrey, Ridgeway, & Morral, 2004), recursive partitioning or tree-based methods (Lee et al., 2010; Setoguchi, Schneeweiss, Brookhart, Glynn, & Cook, 2008), random forests (Lee et al., 2010), and neural networks (Setoguchi et al., 2008) for estimating the propensity score have been examined.”
An Introduction to Machine Learning (2020, methodology): “13 Inspection of the feature importance provides valuable information about the magnitude and the effect of the bias, 6,7 which is recommend to be used for checking the trustworthiness of ML models.”
The role of prediction modeling in propensity score estimation: an evaluation of logistic regression, bCART, and the covariate-balancing propensity score. (2014, methodology): “(19) and Austin (21), we defined the outcome as a linear combination of T and Xi, where i = 1–4, 8–10.”
Matching methods for causal inference: A review and a look forward (2010, background): “The most common for propensity score estimation is logistic regression, although non-parametric methods such as boosted CART and generalized boosted models (gbm) often show very good performance (McCaffrey et al., 2004; Setoguchi et al., 2008; Lee et al., 2009).”
Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies (2006, methodology): “9 Lee et al. (2010) modify the Setoguchi et al. (2008) simulations in that the outcome variable is continuous instead of binary, and they use the simulations to evaluate the performance of propensity score weighting. semiparametrically (Lehrer & Kordas, 2013).”
Representation Learning for Treatment Effect Estimation from Observational Data (2018, background): “Propensity score is the probability that a unit receives the treatment [28, 22].”
Evaluation of subset matching methods and forms of covariate balance (2016, methodology): “In practice, a good alternative for distance driven matching methods may be to estimate the propensity score using a more flexible approach than logistic regression, for example, by using ensemble methods as in [51].”
An introduction to propensity-score methods for reducing confounding in observational studies (2011, neutral): “Although logistic regression appears to be the most commonly used method for estimating the propensity score, the use of bagging or boosting (Lee, Lessler, & Stuart, 2010; McCaffrey, Ridgeway, & Morral, 2004), recursive partitioning or tree-based methods (Lee et al., 2010; Setoguchi, Schneeweiss, Br”
Prognostic score–based balance measures for propensity score methods in comparative effectiveness research (2013, neutral): “First, better covariate balance (at least by some measures) does not always yield less biased effect estimates [3].”