Risk-averse dynamic programming for Markov decision processes

By Andrzej Ruszczynski. Mathematical programming, 2010

558 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Risk-Averse Bayes-Adaptive Reinforcement Learning (2021, background): “Unlike dynamic Markov risk measures [35], history-dependent policies may be required to optimally solve this static CVaR optimisation problem [3, 8, 43].”
Lectures on Stochastic Programming: Modeling and Theory (2009, background): “We thank Stevens Institute of Technology and Rutgers University for granting sabbatical leaves to Darinka Dentcheva and Andrzej Ruszczyński, during which a large portion of this work was written.”
How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics (2017, methodology): “These single period metrics assess the risk of a random cost incurred at time-step k+ 1 from the perspective of time-step k (we refer the reader to [26] for a more thorough introduction to time consistency of risk metrics).”
Distributionally Robust Markov Decision Processes (2010, background): “Indeed, such approach has also been developed in the mathemati cal finance community, usually in the static setup [19, 20].”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “Similar derivations have been done several times for discounted and episodic MDPs (Littman and Szepesvári, 1996; Ruszczyński, 2010; Azar et al., 2011; Rawlik et al., 2012; Asadi and Littman, 2016; Fox et al., 2016), but we are not aware of any work that considers the average-reward case.”