Classes of Multiagent Q-learning Dynamics with epsilon-greedy Exploration

By Michael Wunder, Michael L. Littman, Monica Babes-Vroman. 2010

169 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

OpenSpiel: A Framework for Reinforcement Learning in Games (2019, neutral): “under specific update dynamics [72, 80, 13, 79, 11, 82, 2, 84, 83, 9, 78].”
Learning with Opponent-Learning Awareness (2017, neutral): “An initial step is to understand the dynamics in inﬁnitesimal gradient ascent, similar to what was done by Wunder, Littman, and Babes (2010).”
Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems (2012, background): “In (Gomes & Kowalczyk, 2009; Wunder, Littman, & Babes, 2010), the authors analyse a continuous-time version of decentralized Q-learning.”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “y gradient optimization against worst-case opponents, in a new algorithm called Exploitability Descent [134].17 We refer the interested reader to seminal works about convergence in multiagent domains [136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146]. Note that instead of convergence, some MAL ”
Emergent Communication through Negotiation (2018, background): “…prisoner’s dilemma game, rational agents playing a single-shot version of the game learn to always defect, whereas rational agents playing iterated prisoner’s dilemma can learn more mutually beneﬁcial co-operative strategies under the right conditions (Sandholm & Crites, 1996; Wunder et al., 2010).”
Succinct and Robust Multi-Agent Communication With Temporal Message Control (2020, methodology): “During training, the action is chosen with the -greedy policy [37].”
Game Theory and Multi-agent Reinforcement Learning (2012, background): “A recent overview of multi-agent Q-learning approaches can be found in (Wunder et al, 2010).”
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning (2017, background): “Besides, convergence to a Nash equilibrium situation where no agent has interest in changing policies is generally not guaranteed even for suboptimal equilibria on simple games [27, 18].”
Artificial Intelligence, Algorithmic Pricing, and Collusion (2020, methodology): “Second, for the ε -greedy model of exploration used in this paper, the deterministic dynamics is complicated by the jumps that occur when the Q-values of the best and second-best actions cross, which give raise to hybrid dynamical systems (Gomes and Kowalczyk, 2009; Wunder et al. 2010).”