Robust Markov Decision Processes

By Wolfram Wiesemann, D. Kühn, Berç Rustem. 2013

554 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “increases in the difﬁculty of the problem, in terms of both computational complexity and data requirements. While there has been some recent work on robust MDP solutions which accounts for this issue [34], [35], a more general and sound theoretical and empirical evaluation is still lacking. Moreover”
Safe Model-based Reinforcement Learning with Stability Guarantees (2017, background): “Similarly, robust MDPs maximize rewards when transition probabilities are uncertain [9, 10].”
Safe Policy Improvement by Minimizing Robust Baseline Regret (2016, methodology): “The main difference is the availability of the baseline policy that creates unique challenges for sequential optimization.”
Distributionally Robust Optimization: A Review (2019, background): “[317] study Markov decision processes where the transition Kernel is known.”
Recent advances in robust optimization: An overview (2014, background): “Finally, Wiesemann et al. (2013b) propose a unifying framework for modeling and solving distributionally robust optimization problems based on standardized ambiguity sets that encompass many ambiguity sets from the recent literature as special cases.”
Reinforcement Learning under Model Mismatch (2017, background): “Other probabilistic and risk based settings were studied by [10, 25, 23] which propose various mechanisms to incorporate percentile risk into the model.”
Robust Adversarial Reinforcement Learning (2017, background): “(Wiesemann et al., 2013) present an enhancement to standard MDP that provides probabilistic guarantees to unknown model parameters.”
Regret based Robust Solutions for Uncertain Markov Decision Processes (2013, background): “Recently, techniques have been proposed to deal with dependence of uncertainties [15, 6].”
RAAM: The Benefits of Robustness in Approximating Aggregated MDPs in Reinforcement Learning (2014, methodology): “We focus on constructing aggregate RMPDs with rectangular uncertainty sets (Iyengar, 2005; Wiesemann et al. , 2013) and show that it is possible to use MDP structural properties to reduce RAAM performance loss guarantees compared to (1.2).”
Robust Policy Optimization with Baseline Guarantees (2015, methodology): “A.1 Sampling Bounds [22] showed that L 1 − deviation of the empirical distribution from the true distribution over m distinct events from n samples is bounded as Now consider a ﬁxed state-action pair ( x, a ) and assume that the transition probability (cid:98) P ( x, a ) has been estimated using N…”
Action Robust Reinforcement Learning and Applications in Continuous Control (2019, background): “Furthermore, and to the best of our knowledge, in this line of works, discussion on the existence of strong-duality does not exists (i.e., as Proposition 1 and 4 assert for PR- and NR-MDPs).”
The Importance of Pessimism in Fixed-Dataset Policy Optimization (2020, unknown): “2 is similar to constructing an s-rectangular robust MDP Wiesemann et al. (2013). More recently, approaches have been proposed which go beyond s-rectangular (Goyal & Grand-Clement, 2018), and such approaches likely have natural parallels in implementing value uncertainty functions.”
Contextual Markov Decision Processes (2015, methodology): “Models described by robust MDPs (Nilim & El Ghaoui, 2005; Wiesemann et al., 2013) consider uncertainty in the transitions and rewards.”
Scaling up budgeted reinforcement learning (2019, background): “…the optimisation criteria include other statistics of the performance, such as the worst-case realisation [Iyengar, 2005; Nilim and El Ghaoui, 2005; Wiesemann et al. , 2013], the variance-penalised expectation [Garc´ıa and Fern´andez, 2015; Tamar et al. , 2012], or the Conditional Value-At-Risk…”