Multi-agent Reinforcement Learning in Sequential Social Dilemmas

By Joel Z. Leibo, Vinícius Flores Zambaldi, Marc Lanctot, Janusz Marecki, Thore Graepel. 2017

712 citations recorded by the source corpus.

Matrix games like Prisoner's Dilemma have guided research on social dilemmas for decades. However, they necessarily treat the choice to cooperate or defect as an atomic action. In real-world social dilemmas these choices are temporally extended. Cooperativeness is a property that applies to policies, not elementary actions. We introduce sequential social dilemmas that share the mixed incentive structure of matrix game social dilemmas but also require agents to learn policies that implement their strategic intentions. We analyze the dynamics of policies learned by multiple self-interested independent learning agents, each using its own deep Q-network, on two Markov games we introduce here: 1. a fruit Gathering game and 2. a Wolfpack hunting game. We characterize how learned behavior in each domain changes as a function of environmental factors including resource abundance. Our experiments show how conflict can emerge from competition over shared resources and shed light on how the sequential nature of real world social dilemmas affects cooperation.

Subfields: Artificial Intelligence, Management Science and Operations Research, Sociology and Political Science, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yoshua Bengio
  4. Shimon Whiteson
  5. Mohit Bansal
  6. Ruslan Salakhutdinov
  7. Quoc V. Le
  8. Percy Liang

In Management Science and Operations Research

  1. Shimon Whiteson
  2. Stefano V. Albrecht
  3. Pieter Abbeel
  4. Sergey Levine
  5. Filippos Christianos
  6. Wendelin Böhmer
  7. Yuxi Li
  8. Adam Lerer

In Sociology and Political Science

  1. Shimon Whiteson
  2. Alex Peysakhovich
  3. Adam Lerer
  4. Yuxi Li
  5. Thore Graepel
  6. Adrian Weller
  7. Joel Z. Leibo
  8. Karl Tuyls

In Reinforcement Learning

  1. Shimon Whiteson
  2. Sergey Levine
  3. Pieter Abbeel
  4. Jakob N. Foerster
  5. Yuxi Li
  6. Stefano V. Albrecht
  7. Filippos Christianos
  8. Wendelin Böhmer

In Game & Decision Theory

  1. Shimon Whiteson
  2. Karl Tuyls
  3. Alex Peysakhovich
  4. Adam Lerer
  5. Julien Pérolat
  6. Thore Graepel
  7. Yuxi Li
  8. Marc Lanctot

Citations in the corpus