Collaborative Multiagent Reinforcement Learning by Payoff Propagation

By Jelle R. Kok, Nikos A. Vlassis. 2006

365 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “Sparse cooperative Q-learning (Kok and Vlassis, 2006) is a tabular Q -learning algorithm that learns to coordinate the actions of a group of cooperative agents only in the states in which such coordination is necessary, encoding those dependencies in a coordination graph.”
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (2017, background): “Kok & Vlassis (2006) show that coordination can be learnt by estimating a global Q-function in the classical distributed setting supplemented with a coordination graph.”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “Zhang & Lesser (2011); Kok & Vlassis (2006) also utilize the sparsity and locality of coordination.”
QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, background): “Guestrin et al. (2002b) and Kok & Vlassis (2006) are some of the earlier representative works.”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, background): “Sparse cooperative Q-learning (Kok and Vlassis, 2006) is a tabular Q-learning algorithm that learns to coordinate the actions of a group of cooperative agents only in the states in which such coordination is necessary, encoding those dependencies in a coordination graph.”
Deep Coordination Graphs (2019, methodology): “However, the tabular approach restricts practical application of SCQL to tasks with few agents and small state and action spaces.”
Coordinated Multi-Agent Reinforcement Learning in Networked Distributed POMDPs (2011, neutral): “…terms in (3) and write the update rule for each group l: Q l ( ⃗ h t l , a t l ) = (1−α)Q l ( ⃗ h t l , a t l )+α[r t l +γQ l ( ⃗ h t+1 l , a * l )] (4) Similar to Sparse Cooperative Q-Learning (Kok and Vlassis 2006), update rule in (4) is based on local reward and Q-function, except for a * l .”
Interactive machine learning for health informatics: when do we need the human-in-the-loop? (2016, background): “Moreover, collaborative multi-agent reinforcement learning has a lot of research potential for machine learning [96].”
Exploiting Agent and Type Independence in Collaborative Graphical Bayesian Games (2011, methodology): “4This corresponds to converting an edge-based representation o an agent-based representation (Kok and Vlassis, 2006). of the houses.”
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey (2020, methodology): “[143] Sparce cooperative Q-learning [144] Toll roads Dynamic lane management Personal simulator Density based Ratio based Random search”
Bounded approximate decentralised coordination via the max-sum algorithm (2009, background): “Our point of departure is recent work demonstrating that the max-sum algorithm is a very promising technique for decentralised coordination (and, more generally, constraint reasoning), providing solutions close to optimality while requiring very limited communication overhead and computation [12, 20”
Automated Design of Adaptive Controllers for Modular Robots using Reinforcement Learning (2008, methodology): “More recently there has been much theoretical and algorithmic work in multi-agent reinforcement learning, notably in cooperative reinforcement learning through distributed value functions (Schneider et al. 1999), coordination graphs (Guestrin et al. 2002, Kok & Vlassis 2006 ), and individual reward”
Efficient Distributed Reinforcement Learning through Agreement (2008, background): “In addition to prior work on distributed GAPS [12], cooperation in distributed RL has been addressed in the fully observable domain through distributed value functions [13] and coordination graphs [5, 7].”
Off-Policy Multi-Agent Decomposed Policy Gradients (2020, background): “Sparse cooperative Q-learning (Kok & Vlassis, 2006) learns to coordinate the actions of a group of cooperative agents only in the states where such coordination is necessary.”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, background): “However, ﬁnding suitable joint actions in a MOCoG is also key to ﬁnding, e.g., coverage sets for MOMMDPs (as is also the case for single-objective CoGs and MMDPs [51, 52]).”
Multi-agent deep reinforcement learning: a survey (2021, background): “Recent approaches focus on exploiting dependencies between agents to decompose the reward among the agents with respect to their actual contribution towards the global reward (Kok and Vlassis 2006).”