Multiagent Planning with Factored MDPs

By Carlos Guestrin, Daphne Koller, Ronald E. Parr. 2001

541 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “…of individual value functions Q a ( τ a , u a ; θ a ), one for each agent a , that condition only on individual action-observation histories: Strictly speaking, each Q a is a utility function (Guestrin et al., 2002) and not a value function since by itself it does not estimate an expected return.”
Learning Multiagent Communication with Backpropagation (2016, methodology): “[7] use a s gle larg MDP to control a collection of agent , via a factored message passing fr m w rk whe e t e mes ages are lea ned.”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “The coordination graph (Guestrin et al., 2002b;a; B ¨ ohmer et al., 2020) is a promising approach to achieve scalable centralized learning.”
QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, methodology): “Guestrin et al. (2002b) and Kok & Vlassis (2006) are some of the earlier representative works.”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, background): “Strictly speaking, each Qa is a utility function (Guestrin et al., 2002) and not a value function since by itself it does not estimate an expected return.”
MAVEN: Multi-Agent Variational Exploration (2019, methodology): “[16, 17] use coordination graphs to factor large MDPs for multi-agent systems and propose inter-agent communication arising from message passing on the graphs.”
The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning (2019, background): “In many problems, the decision of an agent is inﬂuenced by those of only a small subset of other agents [9].”
Multi-Agent Common Knowledge Reinforcement Learning (2018, background): “Guestrin et al. (2002b) represent agents’ value functions as a sum of context-speciﬁc value rules that are part of the agents’ ﬁxed a priori common knowledge.”
Efficient Solution Algorithms for Factored MDPs (2003, methodology): “We begin with a concise overview of the MDP framework, and then describe the representation of factored MDPs.”
Coordinated Multi-Agent Reinforcement Learning in Networked Distributed POMDPs (2011, neutral): “A variable elimination algorithm (Guestrin, Koller, and Parr 2001) can be used to compute an optimal solution for this DCOP, but it requires extensive communication and computation (scaling exponentially with the induced width of the agent interaction graph).”
A survey and critique of multiagent deep reinforcement learning (2018, background): “There are some hybrid approaches that learn a centralized but factored Q value function [242, 243].”
ROMA: Multi-Agent Reinforcement Learning with Emergent Roles (2020, background): “To avoid otherwise exponentially large state-action space, factorizing MDPs for multi-agent systems is proposed (Guestrin et al., 2002).”
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control (2019, methodology): “[22] applied variable elimination after passing Q-function as the message, while [17] proposed a max-plus message passing.”
Scalable Planning and Learning for Multiagent POMDPs (2014, methodology): “This evaluation shows that our approach signiﬁcantly outperforms regular (non-factored) POMCP, indicating that FV-POMCP is able to effectively exploit locality of interaction in both settings.”
Reinforcement learning for factored Markov decision processes (2002, background): “Others ([Dean, Givan, and Kim 1998; Schneider, Wong, Moore, and Riedmiller 1999; Koller and Parr 2000; Peshkin, Kim, Meuleau, and Kaelbling 2000; Guestrin, Koller, and Parr 2002]) present algorithms for finding approximate value functions and approximate policies given factored states and actions.”
Reinforcement Learning with Factored States and Actions (2004, background): “There has been comparatively little on dealing with largeaction spaces or factored actions (Dean et al., 1998; Meuleau et al., 1998; Peshkin et al.,1999; Guestrin et al., 2002).”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, background): “r learning this leads to much larger coverage sets. To keep multi-objective multi-agent planning and reinforcement learning tractable in these settings, it is key to exploit so called loose couplings [36,51], i.e., each Multi-Objective Multi-Agent Decision Making 17 agent’s actions only directly aec”