Coordinated Reinforcement Learning

By Carlos Guestrin, Michail G. Lagoudakis, Ronald E. Parr. 2002

450 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Compatible Value Gradients for Reinforcement Learning of Continuous Deep Policies (2015, background): “4 Structural credit assignment for multiagent learning It is interesting to relate our approach to the literature on multiagent reinforcement learning (Guestrin et al., 2002; Agogino and Tumer, 2004, 2008).”
Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, methodology): “Our approach works with only a team reward, and learns the value-decomposition autonomously from experience, and it similarly differs from the approach with coordination graphs (Guestrin et al., 2002) and the max-plus algorithm (Kuyer et al., 2008; van der Pol and Oliehoek, 2016).”
Celebrating Diversity in Shared Multi-Agent Reinforcement Learning (2021, background): “There are other works that investigate into MARL from the perspective of coordination graphs [28–30], communication [31, 32, 15], and role-based learning [17, 33].”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “The coordination graph (Guestrin et al., 2002b;a; B ¨ ohmer et al., 2020) is a promising approach to achieve scalable centralized learning.”
QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, methodology): “Guestrin et al. (2002b) and Kok & Vlassis (2006) are some of the earlier representative works.”
Deep Coordination Graphs (2019, methodology): “An undirected coordination graph (CG, Guestrin et al., 2002a) G = (cid:104)V , E(cid:105) contains a vertex v i ∈ V for each agent 1 ≤ i ≤ n and a set of undirected edges { i, j } ∈ E between vertices v i and v j .”
Value Function Transfer for Deep Multi-Agent Reinforcement Learning Based on N-Step Returns (2019, background): “Earlier work focuses on constructing coordination graphs (CGs) to represent the interactive relationship between agents and conducting learning according to the speciﬁed relationship in each state [Guestrin et al. , 2002a; Guestrin et al. , 2002b; Kok and Vlassis, 2004 ] .”
The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning (2019, methodology): “In many problems, the decision of an agent is influenced by those of only a small subset of other agents [8, 9] and thus the joint action-value function Q(a) can be represented as a factorization, i.”
Coordinated Multi-Agent Reinforcement Learning in Networked Distributed POMDPs (2011, neutral): “Coordinated reinforcement learning based on coordination graphs (Guestrin, Koller, and Parr 2001) has been explored in (Guestrin, Lagoudakis, and Parr 2002; Kok and Vlassis 2006) for factored MDPs.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “To learn complex multiagent interactions some type of abstraction [352] is often needed, for example, factored value functions [353, 354, 242, 243, 355, 356] (see QMIX and VDN in Section 3.”
Learning to Schedule Communication in Multi-agent Reinforcement Learning (2019, background): “Foer-ster et al. (2016); Sukhbaatar et al. (2016); Peng et al. (2017); Guestrin et al. (2002), and Zhang & Lesser (2013) train multiple agents to learn a communication protocol, and have shown that communicating agents achieve better rewards at various tasks.”
Coordinated Multi-Agent Imitation Learning (2017, background): “Many realistic multi-agent settings require coordination among collaborative agents to achieve some common goal (Guestrin et al., 2002; Kok et al., 2003).”
Structured Exploration via Hierarchical Variational Policy Networks (2018, methodology): “Coordination in RL. Multi-agent coordination has been studied in the RL community (e.g. (Guestrin et al., 2002; Kapetanakis & Kudenko, 2002; Chalkiadakis & Boutilier, 2003), for instance, as a method to reduce the instability of multiple agents learning simultaneously using RL.”
Coordinated Deep Reinforcement Learners for Traffic Light Control (2016, methodology): “In other words, unlike earlier work [4, 6], transfer planning does not attempt to minimize the global approximation error of Q̂.”
Cooperative Multi-Agent Learning: The State of the Art (2005, background): “[84] note that in many domains the actions of some agents may be independent.”
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, methodology): “…details on the preprocessing method to derive ut and xit from st, along with the computation of type vectors are provided in Appendix C. Joint action value estimation: We implement GPL’s joint action value model as fully connected Coordination Graphs (Guestrin et al., 2002) (CGs) for three reasons.”
Off-Policy Multi-Agent Decomposed Policy Gradients (2020, methodology): “In value-based MARL, value decomposition (Guestrin et al., 2002b; Castellini et al., 2019) is widely used.”
Cooperative Multi-agent Control Using Deep Reinforcement Learning (2017, background): “[13] later extended this idea and factored the joint value function into a linear combination of local value functions and used message passing to find the joint optimal actions.”