Interactive Q-learning for Quantiles

By Kristin A. Linn, Eric B. Laber, Leonard A. Stefanski. 2017

47 citations recorded by the source corpus.

ABSTRACT A dynamic treatment regime is a sequence of decision rules, each of which recommends treatment based on features of patient medical history such as past treatments and outcomes. Existing methods for estimating optimal dynamic treatment regimes from data optimize the mean of a response variable. However, the mean may not always be the most appropriate summary of performance. We derive estimators of decision rules for optimizing probabilities and quantiles computed with respect to the response distribution for two-stage, binary treatment settings. This enables estimation of dynamic treatment regimes that optimize the cumulative distribution function of the response at a prespecified point or a prespecified quantile of the response distribution such as the median. The proposed methods perform favorably in simulation experiments. We illustrate our approach with data from a sequentially randomized trial where the primary outcome is remission of depression symptoms. Supplementary materials for this article are available online.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “These proposed algorithms, such as interactive Q-learning [114, 132], Advantage-learning [160], penalized Q-learning [212], the augmented multistage outcome-weighted learning [138], the budgeted learning algorithm [43], and the censored Q-learning algorithm [211], were evaluated using data from an…”
Precision Medicine. (2019, background): “There are other notions of optimality including maximizing quantiles of the outcome distribution (Linn et al 2017; Wang et al In press); maximizing efficacy subject to constraints on risk/harm (Linn et al 2015; Laber et al 2018; Wang et al In press); and maximizing the mean outcome subject to cost…”
A Theoretical Analysis of Deep Q-Learning (2019, background): “o et al. (2011); Zhang et al. (2012); Zhao et al. (2012); Goldberg and Kosorok (2012); Nahum-Shani et al. (2012); Goldberg et al. (2013); Schulte et al. (2014); Song et al. (2015);Zhao et al. (2015); Linn et al. (2017);Zhou et al. (2017); Shi et al. (2018); Zhu et al. (2019) and the references there”
Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning (2016, methodology): “Furthermore, GGQ requires modeling a non-smooth function of the data, which creates complications (Laber et al., 2014; Linn et al., 2017).”