Actor-Critic Algorithms for Risk-Sensitive MDPs

By Prashanth L.A., Mohammad Ghavamzadeh. 2013

141 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Open-Ended Learning Leads to Generally Capable Agents (2021, background): “When thinking about general capability and catastrophic failure of policies, the field of robust and risk sensitive reinforcement learning (Borkar and Jain, 2010; Prashanth and Ghavamzadeh, 2013; Tamar et al., 2012) has been analysing variability in obtained rewards to find safer solutions.”
Adversarial Deep Reinforcement Learning in Portfolio Management (2018, unknown): “devise actorcritic algorithms for estimating the gradient and updating the policy parameters in the ascent direction while establishing the convergence of our algorithms to locally risk-sensitive optimal policies [17].”
Optimizing the CVaR via Sampling (2014, methodology): “A mean-variance criterion was considered by Tamar, Di Castro, and Mannor (2012) and Prashanth and Ghavamzadeh (2013).”
A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning (2016, background): “There is also an extensive literature on risk-averse MDP learning, where the variance of the return is often used as a measure [15, 13, 1, 28]; however, an explicit way to estimate the variance of the return for λ > 0 is not given.”
An Analysis of Categorical Distributional Reinforcement Learning (2018, background): “n of the random return have also been considered in the literature, with a variety of approaches proposing algorithms for estimating more complex distributional information [Morimura et al., 2010b,a, Prashanth and Ghavamzadeh, 2013, Tamar 1Contributed during an internship at DeepMind. Proceedings of”
Risk-Constrained Reinforcement Learning with Percentile Risk Criteria (2015, methodology): “rbation stochastic approximation (SPSA) (Spall, 1992). Risk-sensitive RL with expected exponential utility has been considered in Borkar (2001, 2002). More recently, the works in Tamar et al. (2012); Prashanth and Ghavamzadeh (2013) present RL algorithms for several variance-related risk measures, w”
Algorithms for CVaR Optimization in MDPs (2014, methodology): “If we just follow the policy, then we may useνk −β+ γ k (1−α)1{xk = xT }(−sk)+ as an estimate for∇λL(θ, ν, λ) (see (20) and (22) in Algorithm 2).”