A Generalized Reinforcement-Learning Model: Convergence and Applications

By Michael L. Littman, Csaba Szepesvari. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement learning (2019, neutral): “Any element in A can be given by a which is also assumed to be a vector.”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “Unlike previous work on LP formulations for MDPs, we ﬁnd it useful to regard (6) as the primal .”
Optimistic Bayesian Sampling in Contextual-Bandit Problems (2012, background): “In conventional interpretations of similar problems (Littman, 1996; Singh et al., 2000; Sutton and Barto, 1998), there are two major aspects of generating a policy.”
Multiagent learning in the presence of memory-bounded agents (2013, unknown): “There is a significant volume of parallel literature in MAL that does attempt to have the agents learn the game as well [4,15,22,25,27].”
An anti-jamming stochastic game for cognitive radio networks (2011, background): “It is shown that [34] the minimax-Q learning approach converges to the true Q and V values and hence the optimal policy, as long as each action is tried in every state for infinitely many times.”
Comparative evaluation of MAL algorithms in a diverse set of ad hoc team problems (2012, unknown): “For a selection, see [3–6 , 15– 17, 21– 24 ,32 ].”
Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium (2020, background): “games). These results typically assume access to a sampling oracle, and most of 3 them provide convergence guaranteesthat are asymptoticin nature. In particular, underthe tabular setting, the work in Littman and Szepesvari (1996); Littman (2001a,b); Greenwald et al. (2003); Hu and Wellman (2003); Gr”
The Asymptotic Convergence-rate of Q-learning (1997, background): “Just like in (Littman and Szepesvv ari, 1996 ) (see also the extended version (Szepesvv ari and Littman, 1996)) the main idea is to compare Q t with the simpler process ^ Q t+1 (x; a) = (1 t (x; a)) ^ Q t (x; a) + t (x; a)(r t (x; a) + max b Q (y t ; b)): (6) Note that the only (but rather…”
Correlated Q-Learning (2003, background): “Similarly, if one applies multiagent Q-learning to a two-player constant-sum Markov game, it continues to yield global convergence to the unique equilibrium value of the game [12].”
Algorithms for computing strategies in two-player simultaneous move games (2016, unknown): “Initial results provided conditions under which approximate dynamic programming could be used to guarantee convergence to the optimal value function and policies [19].”
Risk-Sensitive Reinforcement Learning (1998, background): “We gratefully acknowledge the work on worst-case dynamic programming and its reinforcement version of Heger (1994a) and Littman and Szepesvári (1996) because they provided the solid foundation from where we started to formulate our theory.”
Algorithms for Sequential Decision Making (1996, methodology): “Portions of this chapter have appeared in earlier papers: \A generalized reinforcement-learning model: Convergence and applications" [97] with Szepesv ari, and \Generalized Markov decision processes: Dynamicprogramming and reinforcement-learning algorithms" [158] with Szepesv ari.”
Neural Networks for Real-Time Traffic Signal Control (2006, background): “The relations (e.g., convergence properties) between some of the online learning techniques such as Q-learning and stochastic approximation theory have been established in [11] and [12].”
A New Softmax Operator for Reinforcement Learning (2016, background): “However, it is non-differentiable (Property 3), and ignores non-maximizing selections (Property 4).”