Q-learning Residual Analysis: Application to The Effectiveness of Sequences of Antipsychotic Medications for Patients with Schizophrenia
By Ashkan Ertefaie, Susan M. Shortreed, Bibhas Chakraborty. 2016
19 citations recorded by the source corpus.
Q‐learning is a regression‐based approach that uses longitudinal data to construct dynamic treatment regimes, which are sequences of decision rules that use patient information to inform future treatment decisions. An optimal dynamic treatment regime is composed of a sequence of decision rules that indicate how to optimally individualize treatment using the patients' baseline and time‐varying characteristics to optimize the final outcome. Constructing optimal dynamic regimes using Q‐learning depends heavily on the assumption that regression models at each decision point are correctly specified; yet model checking in the context of Q‐learning has been largely overlooked in the current literature. In this article, we show that residual plots obtained from standard Q‐learning models may fail to adequately check the quality of the model fit. We present a modified Q‐learning procedure that accommodates residual analyses using standard tools. We present simulation studies showing the advantage of the proposed modification over standard Q‐learning. We illustrate this new Q‐learning approach using data collected from a sequential multiple assignment randomized trial of patients with schizophrenia. Copyright © 2016 John Wiley & Sons, Ltd.
Subfields: Economics and Econometrics, Psychiatry and Mental health, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Economics and Econometrics
In Psychiatry and Mental health
In Statistics and Probability
Citations in the corpus
- Reinforcement Learning in Healthcare: A Survey (2019, methodology): “roblem, and then presented two methods, bootstrap voting and adaptive confidence intervals, for quantifying the evidence in the data for the choices made by the learned optimal policy. Ertefaie et al. [178] accommodated residual analyses into Q-learning in order to increase the accuracy of model fit a”