Scalable Planning and Learning for Multiagent POMDPs

By Chris Amato, Frans Adriaan Oliehoek. 2014

104 citations recorded by the source corpus.

Online, sample-based planning algorithms for POMDPs have shown great promise in scaling to problems with large state spaces, but they become intractable for large action and observation spaces. This is particularly problematic in multiagent POMDPs where the action and observation space grows exponentially with the number of agents. To combat this intractability, we propose a novel scalable approach based on sample-based planning and factored value functions that exploits structure present in many multiagent settings. This approach applies not only in the planning case, but also in the Bayesian reinforcement learning setting. Experimental results show that we are able to provide high quality solutions to large multiagent planning and learning problems.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning (2019, background): “There, we also investigate another approach called the mixture of experts approach [1], which, in the case of one agent per factor, corresponds to independent learners [25].”
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning (2017, background): “In practice, researchers often resort to approximate forms, by sampling or exploiting structure, to ensure good performance due to intractability [41, 2, 68].”
A survey and critique of multiagent deep reinforcement learning (2018, background): “To learn complex multiagent interactions some type of abstraction [352] is often needed, for example, factored value functions [353, 354, 242, 243, 355, 356] (see QMIX and VDN in Section 3.”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, background): “The simplest way to deploy multi-agent collaborative systems is to make each agent have a completely independent learning process (fully decentralized) [1, 37, 48, 59, 67, 68, 90].”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, methodology): “zation algorithm, are proposed inWu et al.(2010) and Wu et al.(2013), respectively. Furthermore, Monte-Carlo tree search has been applied to special classes of Dec-POMDPs, such as multi-agent POMDPs (Amato and Oliehoek, 2015) and multi-robot active perception (Best et al.,2018). In addition, policy ”