Residual Weighted Learning for Estimating Individualized Treatment Rules

By Xiaoping Zhou, N. Mayer-Hamblett, Umer Khan, Michael R. Kosorok. 2015

235 citations recorded by the source corpus.

ABSTRACT Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. (1) The estimated ITR of OWL is affected by a simple shift of the outcome. (2) The rule from OWL tries to keep treatment assignments that subjects actually received. (3) There is no variable selection mechanism with OWL. All of them weaken the finite sample performance of OWL. In this article, we propose a general framework, called residual weighted learning (RWL), to alleviate these problems, and hence to improve finite sample performance. Unlike OWL which weights misclassification errors by clinical outcomes, RWL weights these errors by residuals of the outcome from a regression fit on clinical covariates excluding treatment assignment. We use the smoothed ramp loss function in RWL and provide a difference of convex (d.c.) algorithm to solve the corresponding nonconvex optimization problem. By estimating residuals with linear models or generalized linear models, RWL can effectively deal with different types of outcomes, such as continuous, binary, and count outcomes. We also propose variable selection methods for linear and nonlinear rules, respectively, to further improve the performance. We show that the resulting estimator of the treatment rule is consistent. We further obtain a rate of convergence for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed RWL methods is illustrated in simulation studies and in an analysis of cystic fibrosis clinical trial data. Supplementary materials for this article are available online.

Subfields: Genetics, Molecular Biology, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Balanced Policy Evaluation and Learning (2017, background): “By scaling and constant shifts, one can therefore reduce minimizing ⌧̂ IPW(⇡) over policies ⇡ 2 ⇧ to minimizing a weighted classification loss over classifiers ⇡ 2 ⇧, providing a reduction to weighted classification [7, 45].”
Learning When-to-Treat Policies (2019, methodology): “Whenever Assumption 4 holds, we can use the argument from Lemma 2 of Zhou et al. (2018) to show the rate of convergence in (21) holds uniformly over the whole class Π for the oracle estimator ∆̃(π, π′).”
Efficient Policy Learning (2017, methodology): “…to carry out this optimization, we follow Beygelzimer and Langford (2009), Kitagawa and Tetenov (2015), Zhang et al. (2012), Zhao et al. (2012) and Zhou et al. (2015), and note that π̂DML can also be understood as the empirical risk minimizer in a weighted classification problem: π̂ = argmaxπ∈Π {…”
Interpretable Dynamic Treatment Regimes (2016, methodology): “the popularity of these methods is that the Journal of the American Statistical Association’s Theory and Methods Invited Paper and the Case Studies and Applications Invited Paper at the 2016 Joint Statistical Meetings will feature non-parametric methods for estimating treatment regimes (Zhou et al.,”
Offline Multi-Action Policy Learning: Generalization and Optimization (2018, methodology): “In particular, it is unlikely that the same regret bounds in Swaminathan and Joachims (2015), Zhou et al. (2017) would still hold when the propensities are not known and must be estimated.”
A Theoretical Analysis of Deep Q-Learning (2019, background): “ang et al. (2012); Zhao et al. (2012); Goldberg and Kosorok (2012); Nahum-Shani et al. (2012); Goldberg et al. (2013); Schulte et al. (2014); Song et al. (2015);Zhao et al. (2015); Linn et al. (2017);Zhou et al. (2017); Shi et al. (2018); Zhu et al. (2019) and the references therein. Our work provid”
Optimal trees for prediction and prescription (2018, background): “[123] propose a weighted SVM based approach to learn a classifier that prescribes one of the two treatment options.”