Correlated Q-Learning

By Amy Greenwald, Keith B. Hall. 2003

477 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “In these settings, even the correct solution concept can be difficult to define (Kim et al., 2022; Littman, 2001; Wang and Sandholm, 2002; Bowling, 2004; Greenwald and Hall, 2003; Zinkevich, Greenwald, and Littman, 2006; Letcher, Foerster, Balduzzi, Rocktäschel, and Whiteson, 2019).”
Multi-agent Reinforcement Learning in Sequential Social Dilemmas (2017, background): “The generalsum case is significantly more challenging [31], and algorithms either have strong assumptions or need to either track several different potential equilibria per agent [32, 33], model other players to simplify the problem [34], or must find a Figure 3: Left: Gathering.”
Empirically Evaluating Multiagent Learning Algorithms (2014, methodology): “Correlated-Q (Greenwald and Hall, 2003) does something similar except that it chooses from the set of correlated equilibria using a variety of different selection methods.”
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning (2017, methodology): “Classical techniques collect or approximate extra information such as the joint values [62, 19, 29, 56],”
Multiagent Soft Q-Learning (2018, methodology): “Many methods have been proposed for equilibrium learning (Littman 1994; 2001; Hu and Wellman 2003; Greenwald, Hall, and Serrano 2003), where the agents are trying to learn policies that satisfy some equilibrium concept from Game Theory.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “8 We refer the interested reader to seminal works about convergence in multiagent domains [139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149].”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, unknown): “Other ideas include using joint action-value function [17, 26, 49, 54] and value function factorization [27, 44, 77], addressing the variance problem by a large batch size [5], learning grounded cooperative communication protocols between agents [18, 22, 30, 40, 50, 62, 89].”
Cooperative and Distributed Reinforcement Learning of Drones for Field Coverage (2018, methodology): “Such an agreement can be evaluated at equilibrium, such as Nash equilibrium (NE) [24] or Correlated equilibrium (CE) [31].”
Multi-agent Reinforcement Learning: An Overview (2010, methodology): “Convergence to equilibria is a basic stability requirement [42, 50].”
Cooperation and communication in multiagent deep reinforcement learning (2016, methodology): “There are existing algorithms such as Nash-Q and FF-Q (Hu and Wellman, 2003; Littman, 2001) that are designed for learning Nash Equilibria and correlated equilibria (Greenwald and Hall, 2003).”
Learning to compete, coordinate, and cooperate in repeated games using reinforcement learning (2011, methodology): “In these algorithms, V (s) is estimated using game theoretic concepts, including minimax (Littman 1994), NE (Hu and Wellman 1998; Bowling 2000), correlated equilibria (Aumann 1974; Greenwald and Hall 2003), and the NBS (Qiao et al.”
Cyclic Equilibria in Markov Games (2005, background): “The theoretical and empirical success of such algorithms has led researchers to apply the same approach in general-sum games, in spite of exceedingly weak guarantees of convergence (Hu & Wellman, 1998; Greenwald & Hall, 2003 ).”
Learning of coordination: exploiting sparse interactions in multiagent systems (2009, background): “Subsequent approaches addressed the problem of learning equilibrium policies in multiagent domains (e.g., [4, 7 , 13, 18]).”
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning (2019, background): “One drawback of these approaches is the requirement of computing equilibrium solutions (Greenwald et al. , 2003), as well as non-uniqueness of equilibria which requires some form of coordinated equilibrium selection.”
Game Theory and Multi-agent Reinforcement Learning (2012, methodology): “The approach used in Nash-Q can also be combined with other equilibrium concepts, for example correlated equilibria (Greenwald et al, 2003) or the Stackelberg equilibrium (Kononen, 2003).”
Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning (2019, unknown): “…and joint-action learners (Claus & Boutilier, 1998) require opponents play stationary strategies, Nash-Q (Hu & Wellman, 2003) require all agents play towards the Nash equilibrium, so do Correlated Q -learning (Greenwald et al., 2003), Minimax-Q (Littman, 1994), and Friend-or-foe Q (Littman, 2001).”
Simultaneous Adversarial Multi-Robot Learning (2003, background): “, [Littman, 1994; Greenwald and Hall, 2002].”
A Comprehensive Survey of Multiagent Reinforcement Learning (2008, unknown): “This holds for stability requirements, like convergence to equilibria [42], [54], as well as for adaptation requirements, like rationality [13], [56].”