Deep Inverse Reinforcement Learning for Sepsis Treatment

By Chao Yu, Guoqi Ren, Jiming Liu. 2019

49 citations recorded by the source corpus.

Sepsis is a leading cause of mortality in hospitals, but its optimal treatment strategy still remains unclear. Recent years have witnessed several successful applications of Reinforcement Learning (RL) approaches in sepsis treatment, achieving far more efficient strategies than those by clinicians. To ensure such applications, an explicit reward function encoding medical domain knowledge should be specified beforehand to indicate the goal of learning. However, due to the paucity of clear understanding of sepsis itself, there is still considerable inconsistency in the formulation of reward functions for sepsis treatment. In this poster, we address the reward learning problem in RL for treatment of sepsis, which has been largely neglected by previous studies. A deep inverse RL with Mini-Tree (DIRL-MT) model is proposed to infer the best reward functions from a set of presumably optimal treatment trajectories using retrospective real medical data. In the model, the MT component learns the factors that are most important in influencing the mortality during sepsis treatment, while the DIRL component infers the complete reward function in terms of weights of those factors. Our work shows that PaO2 and PT can play a vital role and should be paid more attention in the design of more efficient treatment strategies for sepsis in the future.

Subfields: Artificial Intelligence, Cardiology and Cardiovascular Medicine, Epidemiology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cardiology and Cardiovascular Medicine

In Epidemiology

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “To address this problem, Yu et al. [250] applied IRL methods in sepsis treatment to infer the reward functions of clinicians during their treatment process.”
MOReL : Model-Based Offline Reinforcement Learning (2020, background): “[12], and has applications in healthcare [33, 34, 35], recommendation systems [36, 37, 38, 39], dialogue systems [40, 19, 41], and autonomous driving [42].”
Supervised-actor-critic reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units (2020, background): “In the past decades, applying RL for more efficient decision-making has become a hot research topic in healthcare domains [2], generating a great breakthrough in treatment of diabetics [3], cancer [4], sepsis [5], and many other diseases [6–8].”
Instabilities of Offline RL with Pre-Trained Neural Representation (2021, background): “…in challenging sequential decision making settings where sample-eﬃciency is crucial [Mandel et al., 2014, Gottesman et al., 2018, Wang et al., 2018, Yu et al., 2019]; it is seeing much recent interest due to the large amounts of oﬄine data already available in numerous scientiﬁc and engineering…”
What are the Statistical Limits of Offline RL with Linear Function Approximation? (2020, methodology): “…as oﬀ-policy methods or batch methods) are a promising methodology to alleviate the sample complexity burden in challenging reinforcement learning (RL) settings, particularly those where sample eﬃciency is paramount [Mandel et al., 2014, Gottesman et al., 2018, Wang et al., 2018, Yu et al., 2019].”