Bayesian Policy Search for Multi-Agent Role Discovery
By Aaron Wilson, Alan Fern, Prasad Tadepalli. 2010
16 citations recorded by the source corpus.
Bayesian inference is an appealing approach for leveraging prior knowledge in reinforcement learning (RL). In this paper we describe an algorithm for discovering different classes of roles for agents via Bayesian inference. In particular, we develop a Bayesian policy search approach for Multi-Agent RL (MARL), which is model-free and allows for priors on policy parameters. We present a novel optimization algorithm based on hybrid MCMC, which leverages both the prior and gradient information estimated from trajectories. Our experiments in a complex real-time strategy game demonstrate the effective discovery of roles from supervised trajectories, the use of discovered roles for successful transfer to similar tasks, and the discovery of roles through reinforcement learning.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, methodology): “To solve this problem, Wilson et al. (2010) use Bayesian inference to learn a set of roles and Wang et al. (2020c) design a specialization objective to encourage the emergence of roles.”
- Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery (2019, background): “A complementary line of work learns role-specific parameters and assignment of roles to agents with unique features, where each role is sustained for an entire episode [43], while our agents can dynamically choose skills multiple times in an episode.”
- Lenient Learning in Independent-Learner Stochastic Cooperative Games (2016, methodology): “Bayesian methods have also been used with policy search in a MARL context (Wilson et al., 2010), where the distributions are instead over policy parameters and possible roles of other agents.”