Average reward reinforcement learning: Foundations, algorithms, and empirical results

By Sridhar Mahadevan. 2004

403 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Industrial and Manufacturing Engineering

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “…Bradtke, Barto, & Kaelbling, 1996; Brafman & Tennenholtz, 2002; Kaelbling, Littman, & Cassandra, 1995; Lagoudakis & Parr, 2003; Maei & Sutton, 2010; Mahadevan, 1996; Meuleau, Peshkin, Kim, & Kaelbling, 1999; Moore & Atkeson, 1993; Morimoto & Doya, 2000; Peng & Williams, 1996; Prokhorov &Wunsch,…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Reinforcement learning for closed-loop propofol anesthesia: a study in human volunteers (2014, background): “Techniques that maximize returns over a window of time, like R-learning (Mahadevan, 1996), may be viable candidates for improving control performance.”
Influencing Long-Term Behavior in Multiagent Reinforcement Learning (2022, neutral): “Regarding the state transition T , we follow the single-agent average reward literature (Mahadevan, 1996; Wei et al., 2019; Wan et al., 2021) and assume communicating states: for every pair of states, there exists a joint policy that transitions from one state to the other state in a finite number o”
A Minimum Relative Entropy Principle for Learning and Acting (2008, methodology): “In Mahadevan (1996), a grid-world is described that is especially useful as a test bed for the analysis of RL algorithms.”
Using continuous action spaces to solve discrete problems (2009, methodology): “We will use the Continuous Actor Critic Learning Automaton (Cacla) reinforcement learning algorithm [2] to approximates a continuous mapping from state space to action space.”
Recent Advances in Hierarchical Reinforcement Learning (2003, background): “However, RL algorithms have also been developed for MDPs with other definitions of return, such as average reward MDPs [39] [62].”
Multiagent learning in the presence of memory-bounded agents (2013, positive): “1 A very common assumption in RL literature while dealing with MDPs in average reward setting [9,24,28] 2 ||∞ is the max norm.”
Reinforcement Learning and Markov Decision Processes (2012, background): “See [Mahadevan(1996)] for a survey on average reward RL.”
Tonic dopamine: opportunity costs and the control of response vigor (2007, positive): “This solution can be found using iterative dynamic programming methods such as " value iteration " (Bertsekas and Tsitsiklis 1996) or approximated through online sampling of the task dynamics and temporal-difference learning (Schwartz 1993; Mahadevan 1996; Sutton and Barto 1998).”
On Ensuring that Intelligent Machines Are Well-Behaved (2017, methodology): “Similarly, when designing a reinforcement learning algorithm, we might define Θ to be a set of policies for a Markov decision process and f(θ) to be an objective function like the expected discounted return [Sutton and Barto, 1998] or average reward [Mahadevan, 1996] of the policy θ.”
Metalearning and neuromodulation (2002, background): “A different form of TD error is derived for average reward reward reinforcement learning (Mahadevan, 1996) by replacing the normalized value ð1 2 gÞVðsðtÞÞ in the discounted TD error (13) by the average reward r”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, neutral): “The optimism expressed about RL in the early surveys (Keerthi and Ravindran , 1994; Kaelbling et al., 1996; Mahadevan, 1996) has been bolstered by several success stories.”
A Reinforcement Learning Algorithm Based on Policy Iteration for Average Reward: Empirical Results with Yield Management and Convergence Analysis (2004, background): “Some of the pioneering work in this area can be attributed to Schwartz (1993) and Mahadevan (1994, 1996a, 1996b) .”
Exponential Family Predictive Representations of State (2007, background): “For example, the agent may seek a policy which maximizes the reward over a finite window into the future, the expected discounted reward over the infinite future, or the long-term average reward (Blackwell, 1962)(see alsoLittman, 1996; Mahadevan, 1996, and references therein).”
Opponent interactions between serotonin and dopamine (2002, methodology): “Theoretical treatments of this case avoid the possibility of divergence by considering either discounted values, in which a reward is worth less the further in the future it is expected, or, to avoid various unsatisfactory aspects of discounting (Mahadevan, 1996), differential values, the summed dif”