Marginal Mean Models for Dynamic Regimes

By Susan A. Murphy, M. J. van der Laan, James M. Robins. 2001

467 citations recorded by the source corpus.

Subfields: Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “Nevertheless, OPE is a popular tool in areas such as online advertising (Li et al., 2010) or healthcare (Murphy et al., 2001) where online evaluation can have significant financial or safety consequences.”
Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning (2020, methodology): “Importance reweighting, or inverse propensity scoring, has been used for oﬀ-policy RL [12,18,28,29,32,39].”
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation (2019, methodology): “This problem is widely encountered in many real-life applications (e.g., Murphy et al., 2001; Li et al., 2011; Bottou et al., 2013; Thomas et al., 2017), where online experiments are expensive and high-quality simulators are difficult to build.”
Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning (2019, background): “OPE is a central problem in both RL and in the closely related dynamic treatment regimes (DTR; Murphy et al., 2001).”
More Robust Doubly Robust Off-policy Evaluation (2018, background): “…et al. 2015a; Li et al. 2015; Jiang & Li 2016; Thomas & Brunskill 2016), and some of the results have been applied to problems in marketing (e.g., Li et al. 2011; Theocharous et al. 2015), healthcare (e.g., Murphy et al. 2001; Hirano et al. 2003), and education (e.g., Mandel et al. 2014; 2016).”
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL (2020, background): “…a good policy, and is a particularly crucial ingredient in settings where interactions are costly or safety is of great importance, such as robotics (Gu et al., 2017; Kalashnikov et al., 2018a), health (Murphy et al., 2001), dialog agents (Jaques et al., 2019), and education (Mandel et al., 2014).”
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes (2019, background): “, 2000) and DTRs (Chakraborty and Moodie, 2013; Murphy et al., 2001; Zhang et al., 2013),3 in RL one often assumes that the MDP model M2 holds.”
Batch Mode Reinforcement Learning based on the Synthesis of Artificial Trajectories (2012, background): “Optimal control problems arise in many real-life applications, such as engineering (Ried-miller 2005), medicine (Robins 1986; Murphy et al. 2001; Murphy 2003) or artiﬁcial intelligence (Sutton and Barto 1998).”
Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (2019, methodology): “IS-based off-policy evaluation methods have also seen lots of interest recently especially for short-horizon problems, including contextual bandits [Murphy et al., 2001; Hirano et al., 2003; Dudík et al., 2011; Wang et al., 2017].”
Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning (2019, background): “OPE is also closely related to the problem of dynamic treatment regimes in the causal inference literature (Murphy et al., 2001).”
PERFORMANCE GUARANTEES FOR INDIVIDUALIZED TREATMENT RULES (2011, methodology): “When X is low dimensional and the best rule within a simple class of ITRs is desired, empirical versions of the Value can be used to construct estimators [22, 26].”
Efficiently Breaking the Curse of Horizon: Double Reinforcement Learning in Infinite-Horizon Processes (2019, background): “OPE is a central problem in both RL and in the closely related dynamic treatment regimes (DTR; Murphy et al., 2001).”
Estimating Response-Maximized Decision Rules With Applications to Breastfeeding (2009, background): “There has been considerable emphasis in recent work on the inverse-weighting approach to estimation (Murphy et al. 2001; Hernán, Lanoy, Costagliola, and Robins 2006; Bembom and van der Laan 2007a).”