Multiagent cooperation and competition with deep reinforcement learning

By Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, Raul Vicente. 2015

1,014 citations recorded by the source corpus.

Evolution of cooperation and competition can appear when multiple adaptive agents share a biological, social, or technological niche. In the present work we study how cooperation and competition emerge between autonomous agents that learn by reinforcement while using only their raw visual input as the state representation. In particular, we extend the Deep Q-Learning framework to multiagent environments to investigate the interaction between two learning agents in the well-known video game Pong. By manipulating the classical rewarding scheme of Pong we show how competitive and collaborative behaviors emerge. We also describe the progression from competitive to collaborative behavior when the incentive to cooperate is increased. Finally we show how learning by playing against another adaptive agent, instead of against a hard-wired algorithm, results in more robust strategies. The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments.

Subfields: Artificial Intelligence, Sociology and Political Science, Statistical and Nonlinear Physics, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Shimon Whiteson
  2. Sergey Levine
  3. Yoshua Bengio
  4. Mohit Bansal
  5. Pieter Abbeel
  6. Percy Liang
  7. Jianfeng Gao
  8. Ruslan Salakhutdinov

In Sociology and Political Science

  1. Shimon Whiteson
  2. Alex Peysakhovich
  3. Adam Lerer
  4. Thore Graepel
  5. Karl Tuyls
  6. Christian Schröder de Witt
  7. Chongjie Zhang
  8. Tonghan Wang

In Statistical and Nonlinear Physics

  1. Ying Nian Wu
  2. Song-Chun Zhu
  3. Josh Tenenbaum
  4. Jianyi Wang
  5. Pieter Abbeel
  6. Rob Fergus
  7. Igor Mordatch
  8. Pushmeet Kohli

In Reinforcement Learning

  1. Shimon Whiteson
  2. Chongjie Zhang
  3. Sergey Levine
  4. Jakob N. Foerster
  5. Tonghan Wang
  6. Tabish Rashid
  7. Christian Schröder de Witt
  8. Pieter Abbeel

In Game & Decision Theory

  1. Shimon Whiteson
  2. Karl Tuyls
  3. Julien Pérolat
  4. Adam Lerer
  5. Alex Peysakhovich
  6. Thore Graepel
  7. Marc Lanctot
  8. Pablo Hernandez-Leal

Citations in the corpus