Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning
By Filippos Christianos, Lukas Schäfer, Stefano V. Albrecht. 2020
230 citations recorded by the source corpus.
Exploration in multi-agent reinforcement learning is a challenging problem, especially in environments with sparse rewards. We propose a general method for efficient exploration by sharing experience amongst agents. Our proposed algorithm, called Shared Experience Actor-Critic (SEAC), applies experience sharing in an actor-critic framework. We evaluate SEAC in a collection of sparse-reward multi-agent environments and find that it consistently outperforms two baselines and two state-of-the-art algorithms by learning in fewer steps and converging to higher returns. In some harder environments, experience sharing makes the difference between learning to solve the task and not learning at all.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Filippos Christianos
- Stefano V. Albrecht
- Georgios Papoudakis
- Lukas Schäfer
- Arrasy Rahman
- Tonghan Wang
- Niklas Höpner
- Chongjie Zhang
In Management Science and Operations Research
- Filippos Christianos
- Stefano V. Albrecht
- Lukas Schäfer
- Georgios Papoudakis
- Arrasy Rahman
- Niklas Höpner
- Chongjie Zhang
- Chenghao Li
Citations in the corpus
- Celebrating Diversity in Shared Multi-Agent Reinforcement Learning (2021, background): “SEAC [38] partially solves this problem by sharing trajectories only for off-policy training.”
- Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks (2020, methodology): “They have already been used in other multi-agent research [Christianos et al., 2020, 2021, Rahman et al., 2021, Papoudakis et al., 2021].”
- Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, background): “Recent multiagent reinforcement learning (MARL) approaches (e.g. Christianos et al., 2020; Foerster et al., 2018; Rashid et al., 2018; Lowe et al., 2017) solve such problems by jointly training a set of agents with shared learning procedures.”