Learning for Decentralized Control of Multiagent Systems in Large, Partially-Observable Stochastic Environments
By Miao Liu, Chris Amato, Emily P. Anesta, J. Griffith, Jonathan P. How. 2016
36 citations recorded by the source corpus.
Decentralized partially observable Markov decision processes (Dec-POMDPs) provide a general framework for multiagent sequential decision-making under uncertainty. Although Dec-POMDPs are typically intractable to solve for real-world problems, recent research on macro-actions (i.e., temporally-extended actions) has significantly increased the size of problems that can be solved. However, current methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. To accommodate more realistic scenarios, when such information is not available, this paper presents a policy-based reinforcement learning approach, which learns the agent policies based solely on trajectories generated by previous interaction with the environment (e.g., demonstrations). We show that our approach is able to generate valid macro-action controllers and develop an expectationmaximization (EM) algorithm (called Policy-based EM or PoEM), which has convergence guarantees for batch learning. Our experiments show PoEM is a scalable learning method that can learn optimal policies and improve upon hand-coded “expert” solutions.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Chris Amato
- Jonathan P. How
- Shayegan Omidshafiei
- Miao Liu
- Kavinayan Sivakumar
- Yisong Yue
- Stephan Zheng
- Shimon Whiteson
Citations in the corpus
- Learning for multi-robot cooperation in partially observable stochastic environments with macro-actions (2017, methodology): “We first discuss the background on Dec-POMDPs and MacDec-POMDPs and then describe the PoEM algorithm.”
- Structured Exploration via Hierarchical Variational Policy Networks (2018, background): “In the multi-agent setting, the RL problem has also been considered as an inference problem in e.g. (Liu et al., 2015; Wu et al., 2013; Liu et al., 2016).”