Policy Gradients Beyond Expectations: Conditional Value-at-Risk

By Aviv Tamar, Yonatan Glassner, Shie Mannor. 2014

44 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Adversarial Training for High-Stakes Reliability (2022, background): “Certification and verified models Like the subfield of certified or verified robust machine learning [46, 47, 48, 21], we attempt to obtain extreme robustness for relatively easy problems.”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Speciﬁcally, the importance sampling [118], [119] is used to improve policy estimation and provide convergence proof for the proposed algorithms.”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, unknown): “Specifically, the importance sampling [21, 265] is used to improve policy estimation, and provide convergence proof for the proposed algorithms.”
Risk-Constrained Reinforcement Learning with Percentile Risk Criteria (2015, methodology): “Note that if we change cost to reward and minimization to maximization, this is exactly the American option pricing problem, a standard testbed to evaluate risk-sensitive algorithms (e.g., see Tamar et al. (2012)).”
Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones (2020, background): “Prior work has studied safety in RL in several ways, including imposing constraints on expected return [1, 34], risk measures [17, 29, 31, 33], and avoiding regions of the MDP where constraint violations are likely [4, 5, 10, 12, 36, 37].”
Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients (2019, methodology): “…and the deﬁnition of conditional expectation yields the ﬁnal result: The extension to the case in which Z is replaced by a scalar reward function R ( τ ) , where τ is generated from a parameterized distribution p ( τ | θ ) , follows the proof in Tamar et al. (2014) without additional adaptation.”
Algorithms for CVaR Optimization in MDPs (2014, methodology): “In comparison to [28], while they develop a PG algorithm for CVaR optimization in stochastic shortest path problems that only considers continuous loss distributions, uses a biased estimator for VaR, is not incremental, and has no comprehensive convergence proof, here we study mean-CVaR optimization”
Policy Gradient Bayesian Robust Optimization for Imitation Learning (2021, background): “However, unlike prior work (Heger, 1994; Shen et al., 2014; Tamar et al., 2014; 2015; Tang et al., 2019; Zhang et al., 2021), which focuses on risk with respect to a known reward function and stochastic transitions, we consider policy optimization when there is epistemic uncertainty over the reward ”