POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning
By Joseph D. Futoma, Michael C. Hughes, Finale Doshi-Velez. 2020
62 citations recorded by the source corpus.
Many medical decision-making tasks can be framed as partially observed Markov decision processes (POMDPs). However, prevailing two-stage approaches that first learn a POMDP and then solve it often fail because the model that best fits the data may not be well suited for planning. We introduce a new optimization objective that (a) produces both high-performing policies and high-quality generative models, even when some observations are irrelevant for planning, and (b) does so in batch off-policy settings that are typical in healthcare, when only retrospective data is available. We demonstrate our approach on synthetic examples and a challenging medical decision-making problem.
Subfields: Artificial Intelligence, Emergency Medical Services, Epidemiology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Ziyun Wang
- Nando de Freitas
- Mohit Bansal
- Yoshua Bengio
- T. Paine
- Quoc V. Le
- Finale Doshi-Velez
In Emergency Medical Services
In Epidemiology
Citations in the corpus
- Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment (2021, background): “Trajectories were initially discretized into hourly bins starting at t = 1 hour into ICU admission and were truncated either at discharge or at t = 72 hours (as patients staying beyond 72 hours tend to be abnormal cases).”
- Hyperparameter Selection for Offline Reinforcement Learning (2020, background): “These methods are useful when it is challenging, risky or expensive to have an arbitrary policy interact with an environment, for example for robotics (Cabi et al., 2019), self-driving cars, health care (Futoma et al., 2020), or dialogue (Jaques et al., 2019).”