Nash Q-Learning for General-Sum Stochastic Games

By Junling Hu, Michael P. Wellman. 2003

1,216 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Graph Learning

Citations in the corpus

Towards the Science of Security and Privacy in Machine Learning (2016, background): “Reinforcement learning: Methods that learn a policy for action over time given sequences of actions, observations, and rewards fall in the scope of reinforcement learning [12], [13].”
Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, background): “As is standard in MARL, the underlying environment is modeled as a Markov game where actions are chosen and executed simultaneously, and new observations are perceived simultaneously as a result of a transition to a new state (Littman, 1994, 2001; Hu and Wellman, 2003; Busoniu et al., 2008).”
Empirically Evaluating Multiagent Learning Algorithms (2014, methodology): “Nash-Q (Hu and Wellman, 2003) learns different Q -functions for itself and its opponents and plays a stage-game Nash equilibrium strategy for the game induced by these Q -values.”
Mean Field Multi-Agent Reinforcement Learning (2018, methodology): “Our work follows the same direction as Littman (1994); Hu & Wellman (2003); Bowling & Veloso (2002) on adapting a Stochastic Game (van der Wal et al., 1981) into the MARL formulation.”
Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems (2012, background): “Joint action learners, on the contrary, are capable of perceiving (a posteriori) their actions and/or their rewards, for instance in (Littman, 2001; Hu & Wellman, 2003; Banerjee et al., 2004) to name a few.”
SoK: Security and Privacy in Machine Learning (2018, background): “, runs of video game play) fall in the scope of reinforcement learning (RL) [13], [14].”
The Mechanics of n-Player Differentiable Games (2018, background): “Procedures that converge to Nash equilibria have been found for restricted game classes: potential games, 2-player zero-sum games and a few others (Hu & Wellman, 2003; Hart & Mas-Colell, 2013).”
Multiagent Soft Q-Learning (2018, methodology): “Many methods have been proposed for equilibrium learning (Littman 1994; 2001; Hu and Wellman 2003; Greenwald, Hall, and Serrano 2003), where the agents are trying to learn policies that satisfy some equilibrium concept from Game Theory.”
Multi-Agent Learning with Policy Prediction (2010, background): “…as requiring an agent to know a Nash Equilibrium (NE) and the strategy of the other players(Bowling and Veloso 2002; Banerjee and Peng 2007; Conitzer and Sandholm 2007), or observe what actions other agents executed and what re-wards they received (Hu and Wellman 2003; Conitzer and Sandholm 2007).”
Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving (2016, methodology): “For example, the minimax-Q learning [20] or the Nash-Q learning [15].”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, background): “Examples of joint action learners are minimax-Q (Littman, 1994), fictitious play and AWESOME (Brown, 1951; Conitzer & Sandholm, 2007), hyper-Q (Tesauro, 2003), and Nash-Q (Hu & Wellman, 2003).”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “games with infinite stages, but the total payoff of agents is the discounted sum [17].”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “Mixed setting is also known as the general-sum game setting, where no restriction is imposed on the goal and relationship among agents (Hu and Wellman, 2003; Littman, 2001).”
Learning in Multi-agent Systems with Sparse Interactions by Knowledge Transfer and Game Abstraction (2015, methodology): “Markov game is widely adopted as the model of multiagent reinforcement learning (MARL) [6, 10, 16].”
Decentralized Learning in Markov Games (2008, background): “In [ 7 ] an alternative view on Markov Games is taken, i.e.”
Multi-agent deep reinforcement learning: a survey (2021, background): “Joint-action learners, however, observe the taken actions of all other actions a-posteriori as shown in Hu and Wellman (2003) and Littman (2001).”
Correlated Q-Learning (2003, methodology): “Littman (2001) proves that FF-Q converges in all general-sum Markov games.”