Statistical Inference in Dynamic Treatment Regimes

By Eric B. Laber, Min Qian, Daniel J. Lizotte, W. Pelham, Susan A. Murphy. 2010

35 citations recorded by the source corpus.

Dynamic treatment regimes are of growing interest across the clinical sciences as these regimes provide one way to operationalize and thus inform sequential personalized clinical decision making. A dynamic treatment regime is a sequence of decision rules, with a decision rule per stage of clinical intervention; each decision rule maps up-to-date patient information to a recommended treatment. We briefly review a variety of approaches for using data to construct the decision rules. We then review an interesting challenge, that of nonregularity that often arises in this area. By nonregularity, we mean the parameters indexing the optimal dynamic treatment regime are nonsmooth functionals of the underlying generative distribution. A consequence is that no regular or asymptotically unbiased estimator of these parameters exists. Nonregularity arises in inference for parameters in the optimal dynamic treatment regime; we illustrate the effect of nonregularity on asymptotic bias and via sensitivity of asymptotic, limiting, distributions to local perturbations. We propose and evaluate a locally consistent Adaptive Confidence Interval (ACI) for the parameters of the optimal dynamic treatment regime. We use data from the Adaptive Interventions for Children with ADHD study as an illustrative example. We conclude by highlighting and discussing emerging theoretical problems in this area.

Subfields: Economics and Econometrics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “The adaptive confidence interval described here provides a confidence interval with guarantees on coverage probability (see Laber et al. 2010) for a full discussion and justification); in particular, a 95% adaptive confidence interval will cover the true Q-value in about 95% of training sets.”
Dynamic treatment regimes: technical challenges and applications (2014, methodology): “(The three-treatment version of the ACI is given by Laber et al. (2010).)”
New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes (2015, methodology): “Efforts have been made to construct confidence intervals for the parameters in the Q-function, with main challenges coming from nonregularity due to the nondifferentiability of the max operator (Robins 2004; Chakraborty, Murphy, and Strecher 2010; Laber et al. 2011).”
Constructing Dynamic Treatment Regimes in Infinite-Horizon Settings (2014, unknown): “Among others, Robins (2004) and Laber et al. (2010) proposed solutions to this issue.”
Interactive Q-learning for Probabilities and Quantiles (2014, methodology): “…is an approximate dynamic programming procedure that requires modeling nonsmooth, nonmonotone transformations of data; this leads to nonregular estimators and complicates the search for models that fit the data well (Robins, 2004; Chakraborty et al., 2010; Laber et al., 2010; Song et al., 2011).”
Q-LEARNING WITH CENSORED DATA (2012, methodology): “The approach closest to our proposal is the censored-Q-learning algo-rithm of Zhao et al. (2011).”
Subgroup Identification and Variable Selection from Randomized Clinical Trial Data. (2013, background): “A number of authors, including Murphy [2002], Lavori and Dawson [2004], Robins [2004], Laan and Petersen [2004], Murphy [2005], Laber et al.”
Set-valued dynamic treatment regimes for competing outcomes (2012, background): “Optimal DTRs have been estimated for chronic conditions including ADHD (Nahum-Shani et al., 2010; Laber et al., 2011; Nahum-Shani et al., 2012), depression (Schulte et al.”
Linear Fitted-Q Iteration with Multiple Reward Functions (2012, methodology): “The methods we have presented do not provide uncertainty information about the learned value function or policy, and although the algorithm is based on linear regression which itself has well-established methods for statistical inference, it is known that even standard single-reward fitted-Q iterati”
Interactive model building for Q-learning (2014, background): “As noted in the introduction, IQ-learning does not alleviate the inherent nonregularity present in sequential decision making problems; see Robins (2004), Laber et al. (2014), and Chakraborty et al. (2010).”