Dynamic treatment regimes: technical challenges and applications

By Eric B. Laber, Daniel J. Lizotte, Min Qian, W. Pelham, Susan A. Murphy. 2014

193 citations recorded by the source corpus.

Dynamic treatment regimes are of growing interest across the clinical sciences because these regimes provide one way to operationalize and thus inform sequential personalized clinical decision making. Formally, a dynamic treatment regime is a sequence of decision rules, one per stage of clinical intervention. Each decision rule maps up-to-date patient information to a recommended treatment. We briefly review a variety of approaches for using data to construct the decision rules. We then review a critical inferential challenge that results from nonregularity, which often arises in this area. In particular, nonregularity arises in inference for parameters in the optimal dynamic treatment regime; the asymptotic, limiting, distribution of estimators are sensitive to local perturbations. We propose and evaluate a locally consistent Adaptive Confidence Interval (ACI) for the parameters of the optimal dynamic treatment regime. We use data from the Adaptive Pharmacological and Behavioral Treatments for Children with ADHD Trial as an illustrative example. We conclude by highlighting and discussing emerging theoretical problems in this area.

Subfields: Economics and Econometrics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “cision-making is to develop effective treatment regimes that can dynamically adapt to the varying clinical states and improve the long-term beneﬁts of patients. Dynamic treatment regimes (DTRs) [72], [73], alternatively named as dynamic treatment policies [74], adaptive interventions [75], or adapti”
Learning Cost-Effective and Interpretable Treatment Regimes (2017, unknown): “Furthermore, recent research in personalized medicine has also focused on developing dynamic treatment regimes [14, 40, 33, 7] where the goal is to learn treatment regimes that maximize outcomes for patients in a given population by recommending a sequence of appropriate treatments over time, based ”
Remote patient monitoring using artificial intelligence: Current state, applications, and challenges (2023, background): “With the sequential decision-making capability, different reinforcement learning schemes are applied to diverse dynamic treatment regimes (Laber et al., 2014) like chronic diseases, mental health diseases, and infectious diseases which need a sequence of decision rules to determine a course of…”
STATISTICAL INFERENCE FOR THE MEAN OUTCOME UNDER A POSSIBLY NON-UNIQUE OPTIMAL TREATMENT STRATEGY (2016, background): “Nonetheless, as we and others have argued in the references above, the value of the estimated rule is also an interesting target of inference [Chakraborty, Laber and Zhao (2014), Laber et al. (2014b), van der Laan and Luedtke (2014a, 2014b)].”
Q-Learning: Theory and Applications (2020, methodology): “In addition to problems of statistical inference, developments for finite Q-learning in statistics have included extensions to nonlinear model classes (Zhao et al. 2009, Laber et al. 2014a, Moodie et al. 2014, Zhang et al. 2015, Xu et al. methods (Zhang et al. 2012, Zhang et al. 2015).”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, background): “RL techniques have shown to help in many healthcare tasks, including dynamic treatment regimes [156], [157], automatic medical diagnosis [158], [159], health resource scheduling [160], [161], and drug discovery and development, [162], [163] etc.”
Q-learning Residual Analysis: Application to The Effectiveness of Sequences of Antipsychotic Medications for Patients with Schizophrenia (2016, methodology): “m-out-of-n bootstrap [36] or ACI [30], may potentially be useful.”
Interactive Q-learning for Quantiles (2017, background): “This leads to nonregular estimators for parameters that index the optimal regime and complicates the search for models that fit the data well since many standard regression modeling diagnostics are invalid (Robins 2004; Chakraborty, Murphy, and Strecher 2010; Laber et al. 2014c; Song et al. 2015).”
Interactive model building for Q-learning (2014, background): “As noted in the introduction, IQ-learning does not alleviate the inherent nonregularity present in sequential decision making problems; see Robins (2004), Laber et al. (2014), and Chakraborty et al. (2010).”
Precision Medicine. (2019, methodology): “…Shortreed and Moodie 2012; Shortreed et al 2014; Kosorok and Moodie 2016); measurement error (Shani et al 2013), and model-building (Biernot and Moodie 2010; Rich et al 2010; Henderson et al 2010; Gunter et al 2011; Lu et al 2013; Laber et al 2014; Song et al 2015; Luedtke and van der Laan 2016b).”
A Theoretical Analysis of Deep Q-Learning (2019, methodology): “7); Agarwal et al. (2019); Fujimoto et al. (2019). Moreover, Q-learning, and reinforcement learning methods in general, have been widely applied to dynamic treatment regimes (DTR) (Chakraborty, 2013; Laber et al., 2014; Tsiatis, 2019), where the goal is to ﬁnd sequential decision rules for individua”