Guidelines for reinforcement learning in healthcare

By Omer Gottesman, Fredrik D. Johansson, Matthieu Komorowski, Aldo A. Faisal, D. Sontag, Finale Doshi-Velez, Leo Anthony Celi. 2019

449 citations recorded by the source corpus.

Subfields: Economics and Econometrics, General Health Professions, Health Informatics, Reinforcement Learning, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In General Health Professions

In Health Informatics

In Reinforcement Learning

In Medical & Health AI

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “A key example of this is in sepsis management, where Gottesman et al. (2019) show that a standard search of the state space would miss important parts of it that could be associated with poor outcomes.”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “Datasets can also be significantly biased towards serious outcomes (Gottesman et al., 2019), since minor cases rarely require treatment, and can lead naïve agents to erroneous conclusions, for example that any drug treatment may cause death simply because it is not prescribed to otherwise healthy in”
A Reinforcement Learning–Based Method for Management of Type 1 Diabetes: Exploratory Study (2019, neutral): “In health care, using medical image and treatment regimen–related information from historical medical data, RL was used for cancer prediction, diagnosis, and prognosis [32,33].”
Reinforcement Learning in Healthcare: A Survey (2019, background): “The above features naturally make RL an attractive solution to constructing efficient policies in various healthcare domains, where the decision making process is usually characterized by a prolonged period or sequential procedure [16].”
MOPO: Model-based Offline Policy Optimization (2020, background): “Reinforcement learning (RL) methods, in contrast, struggle to scale to many real-world applications, e.g., autonomous driving [72] and healthcare [21], because they rely on costly online trial-and-error.”
Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning (2019, background): “This issue has been noted as one of the key limitations for the applicability of RL in medical settings (Gottesman et al., 2019).”
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes (2019, background): “This is quite important, given the practical diﬃculty of evaluation in long horizons (see, e.g. , Gottesman et al., 2019) and given that many RL problems are Markovian.”
Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models (2019, background): “These include issues like confounding and small sample sizes, as discussed in (Gottesman et al., 2019).”
Efficiently Breaking the Curse of Horizon: Double Reinforcement Learning in Infinite-Horizon Processes (2019, background): “This issue has been noted as one of the key limitations for the applicability of RL in medical settings (Gottesman et al., 2019).”
Communication-Efficient Edge AI: Algorithms and Systems (2020, background): “For example, AI technologies have shown promises in healthcare [24], such as detection of heart failure [25] with recurrent neural network (RNN) and decisions about patient treatment [26] with reinforcement learning.”
Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learning (2020, background): “1 Introduction In many emerging application domains for reinforcement learning (RL), exploration is highly limited and simulation unreliable, such as in healthcare (Gottesman et al., 2019).”
Artificial Intelligence for Mental Health and Mental Illnesses: An Overview (2019, background): “There are other ML methods like semi-supervised learning (blend of supervised and unsupervised) [39, 40] and reinforcement learning where the algo-rithm acts as an agent in an interactive environment that learns by trial and error using rewards from its own actions and experiences [41].”
Deep Learning in Medical Imaging (2019, background): “8 The main difference between supervised and unsupervised learnings is whether the training data set has labeled outputs corresponding to input data.”
Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions (2020, background): “Furthermore, many of the common assumptions underlying these theoretical guarantees are usually not met in practice: observational healthcare data, for example, often contains many unobserved confounders (Gottesman et al., 2019a).”
Reinforcement Learning based Recommender Systems: A Survey (2021, background): “This made it possible to apply RL in problems with enormous state and action spaces, including self-driving cars [17, 18], robotics [19], industry automation [20], inance [21], healthcare [22, 23], and RSs [24].”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “An RL approach for decisions making in patient treatment was introduced in [317] to realize safe and risk-conscious healthcare practice.”
Statistically Efficient Off-Policy Policy Gradients (2020, unknown): “A key application is reinforcement learning (RL) for healthcare (Gottesman et al., 2019).”
Is Pessimism Provably Efficient for Offline RL? (2020, background): “Due to such a barrier of sample complexity, it remains notably more challenging to apply online RL in critical domains, e.g., precision medicine (Gottesman et al., 2019) and autonomous driving (Shalev-Shwartz et al., 2016), where interactive data collecting processes can be costly and risky.”