Cyclic Equilibria in Markov Games

By Martin A. Zinkevich, Amy Greenwald, Michael L. Littman. 2005

98 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “In these settings, even the correct solution concept can be difficult to define (Kim et al., 2022; Littman, 2001; Wang and Sandholm, 2002; Bowling, 2004; Greenwald and Hall, 2003; Zinkevich, Greenwald, and Littman, 2006; Letcher, Foerster, Balduzzi, Rocktäschel, and Whiteson, 2019).”
Multi-agent Reinforcement Learning in Sequential Social Dilemmas (2017, background): “The generalsum case is significantly more challenging [31], and algorithms either have strong assumptions or need to either track several different potential equilibria per agent [32, 33], model other players to simplify the problem [34], or must find a Figure 3: Left: Gathering.”
Learning with Opponent-Learning Awareness (2017, background): “[47] explicitly study the convergence dynamics and equilibria of learning in iterated games.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “8 We refer the interested reader to seminal works about convergence in multiagent domains [139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149].”
Game Theory and Multi-agent Reinforcement Learning (2012, background): “Moreover, recent research indicates that a reliance on Q-values alone may not be sufficient to learn an equilibrium policy in arbitrary general sum games (Zinkevich et al, 2006) and new approaches are needed.”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, background): “Cyclic equilibria [29,68,131] have been proposed as a solution concept for games where no stationary equilibrium exists.”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, background): “general-sum modeling requires solving algorithms to either track different potential equilibria for each agent or be able to find a cyclic strategy consisting of multiple policies learned by using different state space sweeps [95], [96].”
Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy (2010, background): “Thus, approximating the infinite-horizon game with a very large finite-horizon game does not necessarily converge (Zinkevich et al., 2006).”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “Moreover, [123] has proved that value-iteration methods fail to find stationary Nash or correlated equilibria for general-sum Markov games.”
Actor-Critic Policy Optimization in Partially Observable Multiagent Environments (2018, background): “Much work in classical MARL focuses on Markov games where the environment is fully observable and agents take actions simultaneously, which in some cases admit Bellman operators [55, 103, 70, 69].”
Influencing Long-Term Behavior in Multiagent Reinforcement Learning (2022, unknown): “Another effective approach to addressing the non-stationarity is learning equilibrium policies that correspond to game-theoretic solution concepts (Littman, 1994; 2001; Wang & Sandholm, 2002; Greenwald & Hall, 2003; Zinkevich et al., 2006).”
A Polynomial-time Nash Equilibrium Algorithm for Repeated Stochastic Games (2008, background): “FolkEgal also has guaranteed polynomial runtime bounds, whereas CE-VI is known to fail to converge in some games (Zinkevich et al., 2005).”