Deep Reinforcement Learning for Sepsis Treatment

By Aniruddh Raghu, Matthieu Komorowski, Imran Ahmed, Leo Anthony Celi, Peter Szolovits, Marzyeh Ghassemi. 2017

200 citations recorded by the source corpus.

Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we propose an approach to deduce treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Our model learns clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. The learned policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.

Subfields: Reinforcement Learning, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Medical & Health AI

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “Q-learning methods on this dataset has been applied to problems such as the treatment of sepsis (Raghu et al., 2017) and optimizing the use of ventilators (Prasad et al., 2017).”
Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation (2020, methodology): “, being near discharge, or (2) a treatment that is taken at that time step [24].”
Deconfounding Reinforcement Learning in Observational Settings (2018, methodology): “1 As pointed out by Raghu et al. (2017a), RL also has advantages over other machine learning algorithms even in the observational setting, especially in two situations: when the optimal treatment strategy is unclear in medical literature (Marik, 2015), and when training examples do not represent…”
Evaluating Reinforcement Learning Algorithms in Observational Health Settings (2018, methodology): “Following Raghu et al. [2017], for every patient, each parameter is aggregated at 4-hour intervals.”
Reinforcement Learning in Healthcare: A Survey (2019, methodology): “[197], [198] examined fully continuous state and action space, where policies are learned directly from”
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation (2018, methodology): “The reasons are: 1) SRL-RNN regards the treatment recommendation as a sequential decision process, reflecting the clinical practice (compared with LEAP and POP-20), and utilizes evaluation signal to infer an optimal treat-ment (compared with LG and BL); 2) SRL-RNN considers the prescriptions of doct”
Key challenges for delivering clinical impact with artificial intelligence (2019, background): “…length of stay and discharge diagnoses [34], predicting future deterioration, including acute kidney injury [35], improving decision-making strategies, including weaning of mechanical ventilation [36] and management of sepsis [37], and learning treatment policies from observational data [38].”
Deep Inverse Reinforcement Learning for Sepsis Treatment (2019, background): “In this poster, we focus on the challenging issue of administration of intravenous (IV) and maximum vasopressor (VP) for sepsis patients [1], [2].”
Reinforcement Learning based Recommender Systems: A Survey (2021, methodology): “DRL-based Methods RLRS Year SR PO RF EB Metrics Dataset Application Code Slate-MDP [112] 2015 SR1 DQN R1 Simulation Reward N/A Wolpertinger [24] 2015 SR1 DDPG R1 Simulation Reward N/A Nemati et al. [113] 2016 SR2 DQN R2 Oline Reward MIMIC Healthcare CEI [114] 2017 SR3 REINFORCE-wb R1 Simulation P, F”
Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units (2019, methodology): “proposed an approach to deduce treatment policies for septic patients by using continuous deep RL methods [22], and Weng et al.”
Model-Based Reinforcement Learning for Sepsis Treatment (2018, background): “This work explores the use of reinforcement learning (RL) to provide medical decision support for sepsis treatment, developing on earlier research [10, 14].”