Estimating Individualized Treatment Rules Using Outcome Weighted Learning

By Ying‐Qi Zhao, Donglin Zeng, A. Rush, Michael R. Kosorok. 2012

805 citations recorded by the source corpus.

There is increasing interest in discovering individualized treatment rules (ITRs) for patients who have heterogeneous responses to treatment. In particular, one aims to find an optimal ITR that is a deterministic function of patient-specific characteristics maximizing expected clinical outcome. In this article, we first show that estimating such an optimal treatment rule is equivalent to a classification problem where each subject is weighted proportional to his or her clinical outcome. We then propose an outcome weighted learning approach based on the support vector machine framework. We show that the resulting estimator of the treatment rule is consistent. We further obtain a finite sample bound for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed approach is demonstrated via simulation studies and an analysis of chronic depression data.

Subfields: Experimental and Cognitive Psychology, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Statistics and Probability

Citations in the corpus

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data (2017, background): “Methods to identify the optimal DTRs have recently received attentions in the statistical community [7], [8], [9], [10], [11], [12], [13], [14], and most of exsiting methods focused on data from randomized clinical trials.”
Learning When-to-Treat Policies (2019, methodology): “Our approach builds most directly on methods for structured policy learning in the static setting (Manski 2004; Zhao et al. 2012; Swaminathan and Joachims 2015; Kitagawa and Tetenov 2018; Athey and Wager 2020).”
Using Decision Lists to Construct Interpretable and Parsimonious Treatment Regimes (2015, methodology): “ters for non-parametric Q-learning are selected by minimizing 10-fold cross-validated prediction error. For OWL, both linear and Gaussian kernels are used and we follow the same tuning strategy as in Zhao et al. (2012). For MCA, we incorporate the eciency augmentation term described in Tian et al. (”
A Simple Method for Detecting Interactions between a Treatment and a Large Number of Covariates (2012, methodology): “Remarks 4 The similar technique can also be used for improving other estimators such as that proposed by Zhao et al. [2012], where the surrogate objective function for the weighted misclassification error can be written in the form of (2.”
Efficient Policy Learning (2017, methodology): “, 2011; Swaminathan and Joachims, 2015), outcome-weighted learning (Chen et al., 2016; Qian and Murphy, 2011; Zhang et al., 2012; Zhao et al., 2012; Zhou et al., 2015), and empirical welfare maximization (Kitagawa and Tetenov, 2015).”
Quasi-oracle estimation of heterogeneous treatment effects (2017, background): “Another research direction that also use ideas from semiparametrics to estimate complex objects is centered on estimating optimal treatment allocation rules (Athey and Wager, 2017; Dud´ık et al., 2011; Luedtke and van der Laan, 2016; Zhang et al., 2012; Zhao et al., 2012).”
Some methods for heterogeneous treatment effect estimation in high dimensions (2017, methodology): “In a novel approach, Zhao et al. (2012) used outcome weighted learning to directly determine individualized treatment rules, skipping the step of estimating individualized treatment eﬀects.”
Interactive Q-learning for Quantiles (2017, methodology): “Note that approaches such as OWL (Zhao et al. 2012) and the methods described in Luedtke and van der Laan (2015) do not require models for m ( H 2 ) , while double robust approaches model this term only to increase efficiency and remain consistent for the optimal regime even when the model is…”
Precision Medicine. (2019, methodology): “Outcome Weighted Learning (OWL, Zhao et al 2012) uses this framework with support vector machines (see, e.g., Chapter 12 of Hastie et al 2009) to estimate an optimal treatment regime; convergence rates for OWL were among the first to establish the mathematical underpinnings of directsearch…”
Dynamic treatment regimes: technical challenges and applications (2014, methodology): “Recent statistical work in this area includes marginal structural mean models (Robins et al., 2008; Orellana et al., 2010), augmented value maximization (Zhang et al., 2012, 2013), and outcome weighted learning (Zhao et al., 2012, 2013).”
Estimation of Optimal Dynamic Treatment Regimes (2014, methodology): “Methods for estimating V(d), include inverse probability-weighting [9,14]; augmented inverse probability-weighting [10,13]; and marginal structural mean models [7].”
CAUSAL INTERPRETATIONS OF BLACK-BOX MODELS (2019, background): “Machine learning algorithms may also help us to discover heterogeneity in causal relations and optimize treatment decision (Hill, 2011, Shortreed et al., 2011, Green and Kern, 2012, Zhao et al., 2012, Wager and Athey, 2018).”
Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes (2012, background): “The outcome may be ascertained following the K th decision, as in the case of CD4 T-cell count at a prespeciﬁed follow-up time in a study of HIV infection (Moodie et al., 2007); or may be a function of information accrued over the entire sequence of decisions, as in Henderson et al. (2010), where ou”
Incorporating Patient Preferences into Estimation of Optimal Individualized Treatment Rules (2017, background): “…2012; Moodie et al., 2012, 2014; Laber et al., 2014; Tian et al., 2014) or policy-search-based (Orellana et al., 2010; Zhang et al., 2012, 2013; Zhao et al., 2012, 2014, 2015; Laber and Zhao, 2015), though this division 18 © 2017, The International Biometric Society is somewhat superficial…”
Policy Learning With Observational Data (2017, background): “Our paper builds on a rich literature at the intersection of econometrics, statistics and computer science on learning structured treatment assignment rules, including Kitagawa and Tetenov (2018), Swaminathan and Joachims (2015) and Zhao, Zeng, Rush, and Kosorok (2012).”
Residual Weighted Learning for Estimating Individualized Treatment Rules (2015, methodology): “Zhao et al. (2012) also reported similar ﬁnding in their simulation studies that there are no large diﬀerences in the performance between OWL-Linear and OWL-Gaussian.”
Estimating Optimal Treatment Regimes from a Classification Perspective (2012, methodology): “This added benefit, however, is not available in the classification method of Zhao et al. (2012) or the previous work on robust estimation by Zhang et al. (2012a).”
Learning Cost-Effective and Interpretable Treatment Regimes (2017, positive): “We compared our framework to the following state-of-the-art treatment recommendation approaches: 1) Outcome Weighted Learning (OWL) [42] 2) Modified Covariate Approach (MCA) [31] 3) Interpretable and Parsimonious Treatment Regime Learning (IPTL) [39].”
Constrained Bayesian Optimization with Noisy Experiments (2017, background): “Related policy optimization problems can be found in medicine (Zhao et al., 2012) and reinforcement learning (Wilson et al.”
Augmented outcome-weighted learning for estimating optimal dynamic treatment regimens (2018, methodology): “To avoid model misspecification in Q-learning, Zhao et al 10 proposed outcome-weighted learning (OWL) to estimate the optimal treatment rules by directly optimizing the expected clinical outcome in a single-stage trial.”