Set-valued dynamic treatment regimes for competing outcomes

By Eric B. Laber, Daniel J. Lizotte, Bradley Ferguson. 2012

73 citations recorded by the source corpus.

Dynamic treatment regimes (DTRs) operationalize the clinical decision process as a sequence of functions, one for each clinical decision, where each function maps up‐to‐date patient information to a single recommended treatment. Current methods for estimating optimal DTRs, for example Q‐learning, require the specification of a single outcome by which the “goodness” of competing dynamic treatment regimes is measured. However, this is an over‐simplification of the goal of clinical decision making, which aims to balance several potentially competing outcomes, for example, symptom relief and side‐effect burden. When there are competing outcomes and patients do not know or cannot communicate their preferences, formation of a single composite outcome that correctly balances the competing outcomes is not possible. This problem also occurs when patient preferences evolve over time. We propose a method for constructing DTRs that accommodates competing outcomes by recommending sets of treatments at each decision point. Formally, we construct a sequence of set‐valued functions that take as input up‐to‐date patient information and give as output a recommended subset of the possible treatments. For a given patient history, the recommended set of treatments contains all treatments that produce non‐inferior outcome vectors. Constructing these set‐valued functions requires solving a non‐trivial enumeration problem. We offer an exact enumeration algorithm by recasting the problem as a linear mixed integer program. The proposed methods are illustrated using data from the CATIE schizophrenia study.

Subfields: Economics and Econometrics, Psychiatry and Mental health, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Psychiatry and Mental health

In Statistics and Probability

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “e is heterogeneity across patient preferences for these outcomes, formation of a single composite outcome that correctly balances the competing outcomes for all patients is not possible. Laber et al. [181] then proposed a method for constructing DTRs for schizophrenia that accommodates competing out”
Estimation of Optimal Dynamic Treatment Regimes (2014, methodology): “Other advantages of Q-learning are as follows: (1) it can be extended to accommodate discrete outcomes [45,50], censored outcomes [49], and competing outcomes [54,55]; (2) measures of goodness of fit and visual diagnostics can be used to assess the quality of the fitted regression models in each sta”
A practical guide to multi-objective reinforcement learning and planning (2021, background): “In medical treatment, we may want to maximise the effectiveness of the treatment, while minimising a variety of side effects [Jalalimanesh et al., 2017, Laber et al., 2014, Lizotte et al., 2010].”
Interactive model building for Q-learning (2014, background): “…a measure of efficacy, the Quick Inventory of Depression Symptomatology score, as the outcome (Rush et al., 2004); side-effects or other competing outcomes could be accommodated using set-valued treatment regimes (Lizotte et al., 2012; Laber et al., 2013) or composite outcomes (Wang et al., 2012).”
Interactive Q-learning for Quantiles (2017, background): “This leads to nonregular estimators for parameters that index the optimal regime and complicates the search for models that fit the data well since many standard regression modeling diagnostics are invalid (Robins 2004; Chakraborty, Murphy, and Strecher 2010; Laber et al. 2014c; Song et al. 2015).”
Precision Medicine. (2019, methodology): “…Shortreed and Moodie 2012; Shortreed et al 2014; Kosorok and Moodie 2016); measurement error (Shani et al 2013), and model-building (Biernot and Moodie 2010; Rich et al 2010; Henderson et al 2010; Gunter et al 2011; Lu et al 2013; Laber et al 2014; Song et al 2015; Luedtke and van der Laan 2016b).”
Incorporating Patient Preferences into Estimation of Optimal Individualized Treatment Rules (2017, background): “Set-valued treatment regimes (Laber et al., 2014; Lizotte and Laber, 2016) map current patient information to a subset of possible treatment options that are not estimated to be uniformly worse across all outcomes; the expectation is that a single treatment will be selected from the recommended set…”
Interactive Q-learning for Probabilities and Quantiles (2014, methodology): “Interactive Q-learning (IQ-learning), developed for the two-stage binary treatment setting, requires modeling only smooth, monotone transformations of the data, thereby reducing problems of model misspecification and nonregular inference (Laber et al., 2014).”