Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

By Ying Liu, Brent R. Logan, Ning Liu, Zhiyuan Xu, Jian Tang, Yanzhi Wang. 2017

117 citations recorded by the source corpus.

In this paper, we propose the first deep reinforce-ment learning framework to estimate the optimal Dynamic Treat-ment Regimes from observational medical data. This framework is more flexible and adaptive for high dimensional action and state spaces than existing reinforcement learning methods to model real life complexity in heterogeneous disease progression and treatment choices, with the goal to provide doctor and patients the data-driven personalized decision recommendations. The proposed deep reinforcement learning framework contains a supervised learning step to predict the most possible expert actions; and a deep reinforcement learning step to estimate the long term value function of Dynamic Treatment Regimes. We motivated and implemented the proposed framework on a data set from the Center for International Bone Marrow Transplant Research (CIBMTR) registry database, focusing on the sequence of prevention and treatments for acute and chronic graft versus host disease. We showed results of the initial implementation that demonstrates promising accuracy in predicting human expert decisions and initial implementation for the reinforcement learning step.

Subfields: Artificial Intelligence, Molecular Biology, Statistics and Probability, Reinforcement Learning, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistics and Probability

In Reinforcement Learning

In Medical & Health AI

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “…applied in high-stakes-decision domains, such as healthcare and personalized medicine, including approaches for dynamic treatment regimes (DTR) (Liu et al., 2017), treatment of sepsis (Komorowski et al., 2018), treatment of chronic disorders including epilepsy (Guez et al., 2008), for HIV…”
Reinforcement Learning in Healthcare: A Survey (2019, methodology): “tive ICU data. RL has been also used in the development of optimal policy for the ordering of lab tests in ICUs [222], [223], and prevention and treatments for graft versus host disease (GVHD) [224], [225] using data set from the Center for International Bone Marrow Transplant Research (CIBMTR) regi”
Artificial Intelligence and the Implementation Challenge (2019, background): “…ML can dynamically adapt survey questions to more quickly identify possible diseases [17], dynamically avoid potential communication breakdowns during speech conversation in the assessment of dementia [18], and even recommend treatments directly when using structured institutional data [19].”
Disease Diagnosis in Smart Healthcare: Innovation, Technologies and Applications (2017, methodology): “The lifelong function of dynamic treatment regimes was estimated using reinforcement learning [155].”
A review of cooperative multi-agent deep reinforcement learning (2019, background): “…recent success on deep reinforcement learning (RL)—super-human level control on Atari games [91], mastering the game of Go [118], chess [119], robotic [64], health care planning [81], power grid [41], routing [97], and inventory optimization [100]—several researches have been focused on deep MARL.”
Accelerating Drugs Discovery with Deep Reinforcement Learning: An Early Approach (2018, background): “as robotics [14, 26], natural language processing [27, 38], or healthcare [31, 39].”