Distributionally Robust Markov Decision Processes

By Huan Xu, Shie Mannor. 2010

298 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “Other fundamental issues in RL research include but are not limited to the credit assignment problem [14], [27], the sampel/space/time complexity [28], [29], function approximation [30], [31], safety [32], [33], robustness [34], [35], and”
Risk-sensitive Inverse Reinforcement Learning via Coherent Risk Models (2017, background): “, [33, 49], as well as [34, 13] where the duality between risk and ambiguity as a result of CRMs is exploited.”
Actor-Critic Algorithms for Risk-Sensitive MDPs (2013, background): “, [12, 7, 24]), and 2) the inherent uncertainty related to the stochastic nature of the system, which is the topic of risk-sensitive MDPs (e.”
Tractable Objectives for Robust Policy Optimization (2012, background): “This objective was explored by Mannor et al. [10], where they concluded that the common approach of computing an optimal policy for the expected MDP, i.e., maximizing V π E[M], results in a biased optimization of the desired value expectation under general transition uncertainty.”
Distributionally Robust Optimization: A Review (2019, background): “Yu and Xu [338] extends the work in Xu and Mannor [329, 328] by forming the ambiguity set of distributions”
How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics (2017, background): “e must consider a range of cost outcomes (in contrast to boolean events). Another popular way to quantify risk in control theory and decision making is through the notion of distributional robustness [9, 34, 30]. Distributional robustness captures the idea that the underlying distribution from which”
Distributionally Robust Convex Optimization (2014, background): “We assume that the ambiguity set P in (3) is representable in the standard form P = P ∈ P0(RP × RQ) : EP [Az̃ +Bũ] = b,P [(z̃, ũ) ∈ Ci] ∈ [pi,pi] ∀i ∈ I , (4) where P represents a joint probability distribution of the random vector z̃ ∈ RP appearing in the constraint function v in (3) and some…”
Adaptive Distributionally Robust Optimization (2019, background): “…for instance, ˇ Z´aˇckov´a 1966, Dupacova 1987, Breton and EI Hachem 1995, Shapiro and Kleywegt 2002, Shapiro and Ahmed 2004), and recently to distributionally robust optimization models (see, for instance, Chen et al. 2007, Chen and Sim 2009, Popescu 2007, Delage and Ye 2010, Xu and Mannor 2012).”
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations (2020, background): “Robust Markov decision process (RMDP) [29, 47] considers the worst case perturbation from transition probabilities, and has been extended to distributional settings [82] and partially observed MDPs [48].”