Inference for Nonregular Parameters in Optimal Dynamic Treatment Regimes

By Bibhas Chakraborty, Victor J. Strecher, Susan A. Murphy. 2009

176 citations recorded by the source corpus.

A dynamic treatment regime is a set of decision rules, one per stage, each taking a patient’s treatment and covariate history as input, and outputting a recommended treatment. In the estimation of the optimal dynamic treatment regime from longitudinal data, the treatment effect parameters at any stage prior to the last can be non-regular under certain distributions of the data. This results in biased estimates and invalid confidence intervals for the treatment effect parameters. In this article, we discuss both the problem of non-regularity, and available estimation methods. We provide an extensive simulation study to compare the estimators in terms of their ability to lead to valid confidence intervals under a variety of non-regular scenarios. Analysis of a data set from a smoking cessation trial is provided as an illustration.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Adaptive Interventions Treatment Modelling and Regimen Optimization Using Sequential Multiple Assignment Randomized Trials (Smart) and Q-Learning (2018, methodology): “in 2010 [79] used double bootstrapping method for estimating the CIs of the regression coefficients in multiple regression model of Q-functions.”
Q-Learning: Theory and Applications (2020, methodology): “As for inference, Chakraborty et al. (2010) conducted simulation studies using various boot-strapping motivated.”
Dynamic treatment regimes: technical challenges and applications (2014, methodology): “We use a contrast encoding for A1 and A2 to allow for a comparison with Chakraborty et al. (2009).”
Interactive Q-learning for Quantiles (2017, background): “This leads to nonregular estimators for parameters that index the optimal regime and complicates the search for models that fit the data well since many standard regression modeling diagnostics are invalid (Robins 2004; Chakraborty, Murphy, and Strecher 2010; Laber et al. 2014c; Song et al. 2015).”
New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes (2015, methodology): “Conducting statistical inference for DTRs is especially important to address questions such as “How much confidence do we have in concluding that the obtained DTR is the best compared to other regimes?” Efforts have been made to construct confidence intervals for the parameters in the Q-function, wi”
Interactive Q-learning for Probabilities and Quantiles (2014, methodology): “…treatment regimes include Q-learning (Watkins, 1989; Watkins and Dayan, 1992; Murphy, 2005a), regularized Q-learning (Moodie and Richardson, 2010; Chakraborty et al., 2010; Song et al., 2011; Goldberg et al., 2013), Interactive Q-learning (Laber et al., 2014), g-estimation (Robins, 2004),…”
Dynamic treatment regimes (2009, unknown): “Inferential methods have been discussed in Robins (2004), Moodie and Richardson (2007) and Chakraborty and Murphy (2009).”
Interactive model building for Q-learning (2014, background): “As noted in the introduction, IQ-learning does not alleviate the inherent nonregularity present in sequential decision making problems; see Robins (2004), Laber et al. (2014), and Chakraborty et al. (2010).”
Assessing Conﬁdence in Policies Learned from Sequential Randomized Trials (2009, methodology): “This problem, a symptom of “non-regularity”, arises in various situations where averages of indicators or other non-smooth functions of the data are estimated using the bootstrap [3, 8].”
Penalized Q-Learning for Dynamic Treatment Regimens (2015, unknown): “This is referred to as the hard-threshold estimator by Chakraborty et al. (2010), who also proposed a soft-threshold estimator and implemented several bootstrap methods.”