Generalized Markov Decision Processes: Dynamic-programming and Reinforcement-learning Algorithms

By Csaba Szepesv Ari, Michael L. Littman. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Perseus: Randomized Point-based Value Iteration for POMDPs (2005, methodology): “In our experiments (see Section 5.2) we implement the backup ′ operator using a number of diﬀerent combinations and analyze their eﬀects.”
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms (2000, background): “However, these are very basic results because they apply only to the lookuptable case, and more importantly because they do not seem to extend naturally to general multi-step on-policy algorithms.”
Algorithms for Sequential Decision Making (1996, methodology): “[2] David H. Ackley and Michael L. Littman.”
The Asymptotic Convergence-rate of Q-learning (1997, methodology): “Putting the pieces together and applying them for t = ^ Q t Q t yields Theorem 3.1.”
A Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms (1999, background): “This idea enables us to prove the convergence of asynchronous stochastic processes whose underlying synchronous process is not of the Robbins-:\,Ionro type (e.g., risk sensitive MDPS, model-based algorithms, etc.) in a unified \vay.”
The Asymptotic Convergence-Rate of Q-learning (1997, methodology): “Q-Iearning is a popular reinforcement learning (RL) algorithm whose convergence is well demonstrated in the literature (Jaakkola et al., 1994; Tsitsiklis, 1994; Littman and Szepesvari, 1996; Szepesvari and Littman, 1996).”
A Generalized Reinforcement-Learning Model: Convergence and Applications (1996, methodology): “Extensions of this result for undiscounted \all-policiesproper" mdps (Bertsekas and Tsitsiklis, 1989), a soft state aggregation learning rule (Singh et al., 1995), and a \spreading" learning rule are given in a more detailed version of this paper (Szepesvv ari and Littman, 1996).”
Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey (2004, background): “The theory results provided in [35], [39] may be helpful to obtain some convergence properties for the EXORL algorithm.”
MDPs: Learning in Varying Environments (2002, methodology): “The following result— using the generalized MDP framework of Szepesvári and Littman (1996)—was proven: if an algorithm converges to the optimal value function in an MDP, then in the corresponding ε-MDP the asymptotic distance of the optimal value function and its approximation is bounded, and the…”