Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games

By Xiaofeng Wang, Tuomas Sandholm. 2002

308 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “In these settings, even the correct solution concept can be difficult to define (Kim et al., 2022; Littman, 2001; Wang and Sandholm, 2002; Bowling, 2004; Greenwald and Hall, 2003; Zinkevich, Greenwald, and Littman, 2006; Letcher, Foerster, Balduzzi, Rocktäschel, and Whiteson, 2019).”
Learning Multiagent Communication with Backpropagation (2016, methodology): “47 We us policy gr ie [33] with a stat specific b s lin for delivering a gradient to the model.”
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning (2017, background): “, there are many update rules for which it converges [13, 25]).”
Coordination and Adaptation in Impromptu Teams (2005, background): “Another very active area of research is learning team behavior, particularly in the reinforcement learning paradigm (Sen, Sekaran, & Hale 1994; Claus & Boutilier 1998; Wang & Sandholm 2002).”
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents (2018, methodology): “It is also worth noting that our model generalizes the team Markov game model for collaborative MARL, see Littman (2001); Wang and Sandholm (2003); Arslan and Y¨uksel (2017), where all agents have individual action sets but share a common payo ﬀ function as in the canonical MMDP.”
AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents (2003, background): “This is also an active area of research in multiagent systems in computer science (e.g., (Littman, 1994; Hu & Wellman, 1998; Claus & Boutilier, 1998; Wang & Sandholm, 2002)).”
Decentralized Learning in Markov Games (2008, background): “In [ 20 ] team Markov Games or MMDPs are also approximated as a sequence of intermediate games.”
Multi-agent Reinforcement Learning: An Overview (2010, background): “In Optimal Adaptive Learning (OAL) , virtual games are constructed on top of each stage game of the stochastic game [136].”
Coordination in multiagent reinforcement learning: a Bayesian approach (2003, background): “[16] X. Wang and T. Sandholm.”
Predicting and Preventing Coordination Problems in Cooperative Q-learning Systems (2007, methodology): “One commonly-used model in games where the agents can see each others’ actions is fictitious play [Claus and Boutilier, 1998] and its variant, adaptive play [Wang and Sandholm, 2002; Young, 1993].”
Influencing Long-Term Behavior in Multiagent Reinforcement Learning (2022, unknown): “Another effective approach to addressing the non-stationarity is learning equilibrium policies that correspond to game-theoretic solution concepts (Littman, 1994; 2001; Wang & Sandholm, 2002; Greenwald & Hall, 2003; Zinkevich et al., 2006).”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, methodology): “We note that this model is also referred to as multi-agent MDPs (MMDPs) in the AI community (Boutilier, 1996; Lauer and Riedmiller, 2000), and Markov teams/team Markov games in the control/game theory community (Yoshikawa, 1978; Ho, 1980; Wang and Sandholm, 2003; Mahajan, 2008).”
A Multiagent Reinforcement Learning Algorithm with Non-linear Dynamics (2008, methodology): “…for modeling agents’ interaction and was used by previous researchers in order to analyze the convergence properties of MARL algorithms (Claus & Boutilier, 1998; Singh et al., 2000; Bowling & Veloso, 2002; Wang & Sandholm, 2003; Bowling, 2005; Conitzer & Sandholm, 2007; Abdallah & Lesser, 2006).”
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications (2020, background): “Even under cooperative MARL wherein all workers aim to achieve the same goal, it may not guarantee the convergence to equilibrium policies without central coordination [85].”
Learning of coordination: exploiting sparse interactions in multiagent systems (2009, background): “Subsequent approaches addressed the problem of learning equilibrium policies in multiagent domains (e.g., [4, 7, 13, 18 ]).”