Risk Sensitive Markov Decision Processes

By S. Marcus, Emmanuel Fernn Andez-Gaucherand, Daniel Hernn Andez-Hernn Andez, S. Coraluppi, P. Fard. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Bayesian Reinforcement Learning with Regret Bounds (2018, background): “(1) For β < 0 this utility function is risk-averse and has been studied extensively in the literature on safety and risk-averse reinforcement learning [35, 83, 52, 53, 48, 50].”
Safe Exploration in Markov Decision Processes (2012, background): “The issue of safety, or risk aversion in MDPs has been addressed before [2, 3, 4, 5, 6, 7, 8], but few of the proposed methods provide strong probabilistic guarantees [6], and they tend to assume uncorrelated ambiguity in the system dynamics.”
A unified view of entropy-regularized Markov decision processes (2017, background): “…and the entailing reinforcement-learning algorithms have been proposed to induce safe exploration (Fox et al., 2016) and risk-sensitive policies (Howard and Matheson, 1972; Marcus et al., 1997; Ruszczyński, 2010), or to model observed behavior of imperfect decision-makers (Ziebart et al., 2010;…”
Risk Aversion in Markov Decision Processes via Near Optimal Chernoff Bounds (2012, methodology): “Our approach is closest to the line of work on exponential utility optimization [12, 13].”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Forexample,fromtheperspectiveofobjectiveoptimiza-tion,severalmethodsconsidercostasoneoftheoptimization objectivestoachievesafety,e.g.,[91],[92],[93],[94],[95],[96], [97],[98],[99].”
Risk-Sensitive Reinforcement Learning (1998, background): “The main idea is to transform the cumulative returns by appropriate utility functions and seek optimal policies with respect to this utility measure (Howard & Matheson, 1972; Pratt, 1964; Coraluppi, 1997; Koenig & Simmons, 1994).”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, unknown): “, [24, 36, 109, 116, 126, 173, 201, 240, 264]; some methods consider safety in RL exploration process by leveraging external knowledge, e.”
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach (2015, background): “Risk-sensitive MDPs [9] address the first aspect by replacing the risk-neutral expectation with a risk-measure of the total discounted cost, such as variance, Value-at-Risk (VaR), or Conditional-VaR (CVaR).”
Policy Gradients with Variance Related Risk Criteria (2012, background): “In the context of MDPs and addressing inherent uncertainty, Howard & Matheson (1972) proposed to use an exponential utility function, where the factor of the exponent controls the risk sensitivity.”
Risk-sensitive Inverse Reinforcement Learning via Coherent Risk Models (2017, background): “For instance, in [24] the authors leverage the exponential (or entropic) risk.”
Action and Perception as Divergence Minimization (2020, methodology): “…overconﬁdence (Jaynes, 1957), Occam’s razor to prevent overﬁtting (Jefferys and Berger, 1992), bounded rationality to halt optimization before reaching the point solution (Ortega and Braun, 2011), and risk-sensitivity to account for model misspeciﬁcation (Pratt, 1964; Howard and Matheson, 1972).”
Particle Value Functions (2017, background): “This is a broadly studied choice that is implied by the assumption that the value function V πT ( s, u ) is additive for deterministic translations of the return (Pratt, 1964; Howard & Matheson, 1972; Coraluppi, 1997).”
Risk-Constrained Reinforcement Learning with Percentile Risk Criteria (2015, background): “Risk-Sensitive MDPs: One of the earliest risk metrics used for risk-sensitive MDPs is the exponential risk metric (1/γ)E [ exp(γZ) ] , where Z represents the cumulative cost for a sequence of decisions (Howard and Matheson, 1972).”
Risk-Sensitive Reinforcement Learning (2013, background): “…e.g., in machine learning (Heger, 1994; Mihatsch and Neuneier, 2002), optimal control (Hern´andez-Hern´andez and Marcus, 1996), operations research (Howard and Matheson, 1972; Borkar, 2002), ﬁnance (Ruszczy´nski, 2010), as well as neuroscience (Nagengast et al., 2010; Braun et al., 2011; Niv et…”