Reinforcement Learning Algorithms for Average-Payoff Markovian Decision Processes

By Satinder Singh. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “…Morimoto & Doya, 2000; Peng & Williams, 1996; Prokhorov &Wunsch, 1997; Rummery &Niranjan, 1994; Santamaría, Sutton, & Ram, 1997; Schwartz, 1993; Singh, 1994; Sutton & Barto, 1998; Sutton, Szepesvári, & Maei, 2008; Tsitsiklis & van Roy, 1996; van Hasselt, 2012; Watkins, 1989; Watkins & Dayan,…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “Some of the initial works in bounding the behavior of greedy policies obtained via function approximation are Williams and Baird (1993) and Singh and Yee (1994). TD methods using function approximation are known to converge (Sutton, 1984, 1992).”
Average reward reinforcement learning: Foundations, algorithms, and empirical results (2004, background): “Other more recent work include that of Singh ( Singh, 1994b ), Tadepalli and Ok (Tadepalli & Ok, 1994), and Mahadevan (Mahadevan, 1994).”
An Average-Reward Reinforcement Learning Algorithm for Computing Bias-Optimal Policies (1996, methodology): “While several researchers have proposed ARL algorithms [15, 16, 20], none of these algorithms is guaranteed to yield bias-optimal policies in general.”
A Reinforcement Learning Method for Maximizing Undiscounted Rewards (1993, background): “• Modified R-learning, [Singh, AAAI1994] • A Model-based Algorithm for Bias-optimal, [Mahadevan, ML1996] • H-Learning, [Tadepalli, AI1998] • SMART, [Das, ManagementScience1999] • Relaxed SMART, [Gosavi, ML2004] • Q-P-Learning, [Gosavi, ML2004] • G-Learning, [Gao, SMC2006]”
Learning Without State-Estimation in Partially Observable Markovian Decision Processes (1994, background): “Average Payo POMDPs: A second policy evalu-ation criterion that has been studied in the MDP liter-ature (e.g., Bertsekas, 1987) and more recently in theRL literature (Schwartz, 1993; Singh, 1994) is the av-erage payo per time step criterion.”
Learning Algorithms for Markov Decision Processes with Average Cost (2001, methodology): “Singh [23] presents two Q-learning algorithms for the average cost problem: one is a slight modification of Schwartz’s algorithm which updates the estimate of optimal cost at every step.”
Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts (2011, methodology): “The optimization criterion of infinite-horizon average-cost has been studied in the past two decades [20, 10, 13, 21]; it was used for policy search [3], and successfully applied to gait optimization [23].”
Scaling Up Average Reward Reinforcement Learning by Approximating the Domain Models and the Value Function (1996, background): “Recently, there has been grow-ing interest in Average-reward Reinforcement Learn-ing (ARL) (Schwartz, 1993; Singh, 1994; Mahadevan,1996; Boutilier & Puterman, 1995; Tadepalli & Ok,1994).”
A Minimum Relative Entropy Principle for Learning and Acting (2008, background): “…reward ρ and the optimal relative Q-values Q(x, a) for each state-action pair (x, a) that are solutions to the following system of non-linear equations (Singh, 1994): for any state x ∈ X and action a ∈ A, Q(x, a) + ρ = r(x, a) + ∑ y∈X Pr(x′|x, a) [ max a′ Q(x′, a′) ] = r(x, a) +Ex′ [ max a′ Q(x′,…”
Truncating Temporal Diierences: on the Eecient Implementation of Td for Reinforcement Learning (1995, methodology): “Other reinforcement learning performance measures have also been considered Heger, 1994; Schwartz, 1993; Singh, 1994, but in this work we limit ourselves exclusively to the performance measure speciied by Equation 1.”