Multi-Objective Markov Decision Processes for Data-Driven Decision Support

By Daniel J. Lizotte, Eric B. Laber. 2016

64 citations recorded by the source corpus.

We present new methodology based on Multi-Objective Markov Decision Processes for developing sequential decision support systems from data. Our approach uses sequential decision-making data to provide support that is useful to many different decision-makers, each with different, potentially time-varying preference. To accomplish this, we develop an extension of fitted-Q iteration for multiple objectives that computes policies for all scalarization functions, i.e. preference functions, simultaneously from continuous-state, finite-horizon data. We identify and address several conceptual and computational challenges along the way, and we introduce a new solution concept that is appropriate when different actions have similar expected outcomes. Finally, we demonstrate an application of our method using data from the Clinical Antipsychotic Trials of Intervention Effectiveness and show that our approach offers decision-makers increased choice by a larger class of optimal policies.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “To solve these issues, the authors [141] then proposed the non-deterministic multi-objective FIQ algorithm, which computed policies for all preference functions simultaneously from continuous-state, finite-horizon data.”
The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making (2018, methodology): “Further, multiple objective optimal treatments were explored [10] with non-deterministic fitted-Q to provide guidelines on antipsychotic drug treatments for schizophrenia patients.”
Monte Carlo Tree Search: a review of recent modifications and applications (2021, methodology): “(2015) introduce the k-Rejection Multi-Agent Markov Decision Process k-RMMDP model in the context of real-world disaster emergency responders.”
Incorporating Patient Preferences into Estimation of Optimal Individualized Treatment Rules (2017, background): “Set-valued treatment regimes (Laber et al., 2014; Lizotte and Laber, 2016) map current patient information to a subset of possible treatment options that are not estimated to be uniformly worse across all outcomes; the expectation is that a single treatment will be selected from the recommended set…”
An Optimal Policy for Patient Laboratory Tests in Intensive Care Units (2018, unknown): “Actions are further filtered for consistency: we might consider feature consistency to be defined as rewards being linear in each feature space [15].”
Precision Medicine. (2019, background): “…treatment regimes which recommend a set of ‘acceptable’ treatments given a patient’s history (Fard 2009; Lizotte et al 2012; Laber et al 2014; Lizotte and Laber 2016; Wu 2016); and methods where a primary outcome is maximized while a secondary outcome is constrained to be within an…”