Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

By Ryan J. Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, Igor Mordatch. 2017

6,086 citations recorded by the source corpus.

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Subfields: Artificial Intelligence, Computer Science Applications, LLM Agents & Tool Use, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Pieter Abbeel
  5. Ruslan Salakhutdinov
  6. Jianfeng Gao
  7. Quoc V. Le
  8. Percy Liang

In Computer Science Applications

  1. Shimon Whiteson
  2. Stefano V. Albrecht
  3. Filippos Christianos
  4. Chongjie Zhang
  5. Wendelin Böhmer
  6. Georgios Papoudakis
  7. Christian Schröder de Witt
  8. Tonghan Wang

In LLM Agents & Tool Use

  1. Shimon Whiteson
  2. Zongqing Lu
  3. Chongjie Zhang
  4. Jiechuan Jiang
  5. Wendelin Böhmer
  6. Richard Ryan Williams
  7. Jakob N. Foerster
  8. Stefano V. Albrecht

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Shimon Whiteson
  4. Chelsea Finn
  5. Yuxi Li
  6. Jakob N. Foerster
  7. Shixiang Shane Gu
  8. Chongjie Zhang

Citations in the corpus