Incorporating causal factors into reinforcement learning for dynamic treatment regimes in HIV

By Chao Yu, Yinzhao Dong, Jiming Liu, Guoqi Ren. 2019

34 citations recorded by the source corpus.

BackgroundReinforcement learning (RL) provides a promising technique to solve complex sequential decision making problems in health care domains. However, existing studies simply apply naive RL algorithms in discovering optimal treatment strategies for a targeted problem. This kind of direct applications ignores the abundant causal relationships between treatment options and the associated outcomes that are inherent in medical domains.MethodsThis paper investigates how to integrate causal factors into an RL process in order to facilitate the final learning performance and increase explanations of learned strategies. A causal policy gradient algorithm is proposed and evaluated in dynamic treatment regimes (DTRs) for HIV based on a simulated computational model.ResultsSimulations prove the effectiveness of the proposed algorithm for designing more efficient treatment protocols in HIV, and different definitions of the causal factors could have significant influence on the final learning performance, indicating the necessity of human prior knowledge on defining a suitable causal relationships for a given problem.ConclusionsMore efficient and robust DTRs for HIV can be derived through incorporation of causal factors between options of anti-HIV drugs and the associated treatment outcomes.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chao Yu
2. Jiming Liu
3. Shamim Nemati
4. Guoqi Ren

In Statistics and Probability

1. Chao Yu
2. Jiming Liu
3. Shamim Nemati

Citations in the corpus

• Reinforcement Learning in Healthcare: A Survey (2019, methodology): “[149] proposed a causal policy gradient algorithm and evaluated it in the treatment of HIV in order to facilitate the final learning performance and increase explanations of learned strategies.”
• Deep Inverse Reinforcement Learning for Sepsis Treatment (2019, methodology): “In the DIRL component, Dueling Double Deep Q Network (DDDQN) [6], Bayesain inverse RL [7], [8] and Stochastic gradient descent [9], [10] methods are applied to infer the optimal reward function, in terms of optimal weights in these key influencing factors, based on treatment actions of the clinician”