Decentralized planning under uncertainty for teams of communicating agents
By Matthijs T. J. Spaan, Geoffrey J. Gordon, Nikos A. Vlassis. 2006
75 citations recorded by the source corpus.
Decentralized partially observable Markov decision processes (DEC-POMDPs) form a general framework for planning for groups of cooperating agents that inhabit a stochastic and partially observable environment. Unfortunately, computing optimal plans in a DEC-POMDP has been shown to be intractable (NEXP-complete), and approximate algorithms for specific subclasses have been proposed. Many of these algorithms rely on an (approximate) solution of the centralized planning problem (i.e., treating the whole team as a single agent). We take a more decentralized approach, in which each agent only reasons over its own local state and some uncontrollable state features, which are shared by all team members. In contrast to other approaches, we model communication as an integral part of the agent's reasoning, in which the meaning of a message is directly encoded in the policy of the communicating agent. We explore iterative methods for approximately solving such models, and we conclude with some encouraging preliminary experimental results.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shimon Whiteson
- Jakob N. Foerster
- Sergey Levine
- Josh Tenenbaum
- Pieter Abbeel
- Chuang Gan
- Yilun Du
- Nando de Freitas
In Management Science and Operations Research
- Shimon Whiteson
- Stefan J. Witwicki
- Francisco S. Melo
- Yannis Assael
- Jakob N. Foerster
- Matthijs T. J. Spaan
- Chuang Gan
- Yilun Du
Citations in the corpus
- Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks (2016, methodology): “Furthermore, planning-based RL methods have been employed to include messages as an integral part of the multi-agent reinforcement learning challenge (Spaan et al., 2006).”
- QueryPOMDP: POMDP-Based Communication in Multiagent Systems (2011, methodology): “Round-robin policy optimization has been used to learn communication primitives in Dec-POMDPs whose base models are transition and observation independent [12], but which are coupled through the communication actions agents can choose to execute.”
- Building Cooperative Embodied Agents Modularly with Large Language Models (2023, background): “Our setting can be defined as an extension of the decentralized partially observable Markov decision process (DEC-POMDP) (Bernstein et al., 2002; Spaan et al., 2006; Goldman & Zilberstein, 2003), which can be formalized by ( n, S, { Σ i } , { A i } , { O i } , T, G, R, γ, h ) , where n denotes the…”