Risk-Averse Bayes-Adaptive Reinforcement Learning
By Marc Rigter, Bruno Lacerda, Nick Hawes. 2021
48 citations recorded by the source corpus.
In this work, we address risk-averse Bayes-adaptive reinforcement learning. We pose the problem of optimising the conditional value at risk (CVaR) of the total return in Bayes-adaptive Markov decision processes (MDPs). We show that a policy optimising CVaR in this setting is risk-averse to both the parametric uncertainty due to the prior distribution over MDPs, and the internal uncertainty due to the inherent stochasticity of MDPs. We reformulate the problem as a two-player stochastic game and propose an approximate algorithm based on Monte Carlo tree search and Bayesian optimisation. Our experiments demonstrate that our approach significantly outperforms baseline approaches for this problem.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matthew Riemer
- Irina Rish
- Khimya Khetarpal
- Doina Precup
- Sergey Levine
- Joelle Pineau
- Eric Eaton
- Chelsea Finn
In Management Science and Operations Research
- Doina Precup
- Irina Rish
- Khimya Khetarpal
- Matthew Riemer
- Sergey Levine
- Ofir Nachum
- Aviral Kumar
- George Tucker
Citations in the corpus
- Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Current work also typically assumes a stationary task state distribution for meta-learning (Doshi-Velez and Konidaris, 2013; Wang et al., 2016; Duan et al., 2016; Zintgraf et al., 2018; Rakelly et al., 2019; Zintgraf et al., 2019; Humplik et al., 2019; Fakoor et al., 2019; Ortega et al., 2019; Perez”