Scaling Up Robust MDPs using Function Approximation

By Aviv Tamar, Shie Mannor, Huan Xu. 2014

147 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Conservative Q-Learning for Offline Reinforcement Learning (2020, background): “Robust MDPs [24, 50, 56, 44] have been a popular theoretical abstraction for offline RL, but tend to be highly conservative in policy improvement.”
Challenges of Real-World Reinforcement Learning (2019, background): “Recently, a number of works have sur-faced that have shown this formulation to yield robust policies that are agnostic to a range of perturbations in the environment (Tamar et al., 2014; Mankowitz et al., 2018; Shashua & Mannor, 2017).”
Unicorn: Continual Learning with a Universal, Off-policy Agent (2018, background): “Additions include optimizing for risk-aware, [46], robust [45, 27] as well as exploratory [6, 31] objectives to yield improved performance.”
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis (2021, background): “Recently, a number of works have surfaced that have shown this formulation to yield robust policies that are agnostic to a range of perturbations in the environment (Tamar et al. 2014; Mankowitz et al. 2018a; Shashua and Mannor 2017; Derman et al. 2018, 2019; Mankowitz et al. 2019).”
An empirical investigation of the challenges of real-world reinforcement learning (2020, background): “Recently, a number of works have surfaced that have shown this formulation to yield robust policies that are agnostic to a range of perturbations in the environment [Tamar et al. 2014; Mankowitz et al. 2018a; Shashua and Mannor 2017; Derman et al. 2018; 2019; Mankowitz et al. 2019].”
Adversarial Policies: Attacking Deep Reinforcement Learning (2019, background): “In robust MDPs, the transition function is chosen adversarially from an uncertainty set (Bagnell et al., 2001; Tamar et al., 2014).”
Learning Robust Options (2018, methodology): “As stated in the background section, RPVI converges to a ﬁxed point.”
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design (2020, methodology): “Minimax training has been used in the controls literature [37, 3, 49], the Robust MDP literature [4, 16, 26, 39], and”
Reinforcement Learning under Model Mismatch (2017, methodology): “…the transition matrices to lie in some uncertainty set and proposed a dynamic programming algorithm to solve the robust MDP. Recent work by [26] extended the robust MDP framework to the function approximation setting where under a technical assumption the authors prove convergence to an…”