Rational and Convergent Learning in Stochastic Games

By Michael H. Bowling, Manuela M. Veloso. 2001

359 citations recorded by the source corpus.

Subfields: Statistical Learning Theory, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

In Game & Decision Theory

Citations in the corpus

Empirically Evaluating Multiagent Learning Algorithms (2014, methodology): “ite of game generators, generating large game sets is now easy. Indeed, Nudelman et al. (2004) also performed one of the largest previous MAL experiments, using three MAL algorithms (minimax-Q, WoLF (Bowling and Veloso, 2001), and Q-learning)on 100 game instances from each of thirteen distributions.”
Mean Field Multi-Agent Reinforcement Learning (2018, background): “In line with (Bowling & Veloso, 2001; 2002), we argue that to better evaluate a multi-agent learning algorithm, on top of the convergence guarantee, discussion on property of Rationality is also needed.”
Deep Reinforcement Learning from Self-Play in Imperfect-Information Games (2016, neutral): “Neural Fitted Q Iteration (NFQ) (Riedmiller, 2005) and Deep Q Network (DQN) (Mnih et al., 2015) are extensions of FQI that use neural network function approximation with batch and online updates respectively.”
Multiagent learning using a variable learning rate (2002, background): “1 This is an extended version of the papers presented at IJCAI-2001, Seattle, WA, USA [7] and ICML-2001, Williamstown, MA, USA [8].”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, background): “For example, Bowling and Veloso (2001) proposed policy hill climbing with the win or learn fast heuristic (WoLF-PHC), and show that the algorithm is rational and convergent in multi-agent domains.”
Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning (2019, neutral): “Typical examples include the series of WoLF models (Bowling, 2005; Bowling & Veloso, 2001a; 2002) or the Nash-Q learning (Hu & Wellman, 2003), both of which require the Nash Equilibrium at each stage game to update the Q-function.”
Opportunities for multiagent systems and multiagent reinforcement learning in traffic control (2009, background): “In this kind of environment it may make sense to evaluate MARL algorithms by the criteria proposed in [13], namely convergence to a stationary policy, and convergence to a best response if the opponent converges to a stationary policy.”
Run the GAMUT: a comprehensive approach to evaluating game-theoretic algorithms (2004, methodology): “Experimental Setup We used three learning algorithms: Minimax-Q [10], WoLF [2], and a version of the original Q-learning algorithm for single agent games [22] modified for use by an individual player in a repeated game setting.”
An Overview of Cooperative and Competitive Multiagent Learning (2005, neutral): “environments are [148, 25, 188, 37, 158, 184, 143, 47].”
Stable Opponent Shaping in Differentiable Games (2018, background): “These include policy prediction in two-player two-action bimatrix games (Zhang & Lesser, 2010); WoLF in two-player two-action games (Bowling & Veloso, 2001); AWESOME in repeated games (Conitzer & Sandholm, 2007); Optimistic Mirror Descent in two-player bilinear zero-sum games (Daskalakis et al.,…”
Accelerating Reinforcement Learning through Implicit Imitation (2003, background): “A general solution requires the integration of imitation into more general models for multiagent RL based on stochastic or Markov games (Littman, 1994; Hu & Wellman, 1998; Bowling & Veloso, 2001 ).”
A Theoretical Analysis of Deep Q-Learning (2019, background): “ning. Subsequently, for Markov games, various online algorithms are also proposed with theoretical guarantees. These work consider either the tabular case or linear function approximation. See, e.g., Bowling (2001); Conitzer and Sandholm (2007); Prasad et al. (2015); Wei et al. (2017); P´erolat et a”
Learning of coordination: exploiting sparse interactions in multiagent systems (2009, background): “Subsequent approaches addressed the problem of learning equilibrium policies in multiagent domains (e.g., [ 4 , 7, 13, 18]).”
Evolutionary Tournament-Based Comparison of Learning and Non-Learning Algorithms for Iterated Games (2007, background): “…Simulation Introduction 1.1 Learning and reasoning in single or multi-stage games has been an active area of research in multiagent systems (Banerjee, Sen, & Peng 2001;Bowling & Veloso 2001; Claus & Boutilier 1998; Greenwald & Hall 2003; Hu &Wellman 1998; Littman 1994, 2001; Littman & Stone 2001).”
Approximate Dynamic Programming for Two-Player Zero-Sum Markov Games (2015, methodology): “As for MDPs, SGs constitute a model for MARL (Littman, 1994) and have been largely studied in past years (Hu & Wellman, 2003; Greenwald et al., 2003; Bowling & Veloso, 2001).”
Multi-Agent Reinforcement Learning:a critical survey (2003, methodology): “We will concentrate on what might be called the “Bellman heritage” in multi-agent RL – work that is based on Q-learning [Watkins and Dayan1992], and through it on the Bellman equations [Bellman1957].”
Classes of Multiagent Q-learning Dynamics with epsilon-greedy Exploration (2010, methodology): “The simple IQL- GHDS can be represented as an automaton whose nodes are four complete and separate dynamical system flows and where transitions between the nodes, or index states, must be taken when certain conditions along them are met.”
Correlated Q-Learning (2003, background): “Bowling (2001) put forth two desiderata for multiagent learning algorithms: rationality and convergence.”
Reinforcement learning in robotic applications: a comprehensive survey (2021, background): “Here, stability and adaptiveness ensure the stationary policy and improvement in performance when agents change their policies respectively (Bowling and Veloso 2002; Greenwald et al. 2003; Hu and Wellman 2003; Bowling and Veloso 2001).”
Cooperative Multi-Agent Learning: The State of the Art (2005, background): “Stone and Veloso [255] argue that unrestricted communication reduces a multi-agent system to something isomorphic to a single-agent system.”