Emergent Complexity via Multi-Agent Competition

By Trapit Bansal, Jakub W. Pachocki, Szymon Sidor, Ilya Sutskever, Igor Mordatch. 2017

435 citations recorded by the source corpus.

Reinforcement learning algorithms can train agents that solve problems in complex, interesting environments. Normally, the complexity of the trained agent is closely related to the complexity of the environment. This suggests that a highly capable agent requires a complex environment for training. In this paper, we point out that a competitive multi-agent environment trained with self-play can produce behaviors that are far more complex than the environment itself. We also point out that such environments come with a natural curriculum, because for any skill level, an environment full of agents of this level will have the right level of difficulty. This work introduces several competitive multi-agent environments where agents compete in a 3D world with simulated physics. The trained agents learn a wide variety of complex and interesting skills, even though the environment themselves are relatively simple. The skills include behaviors such as running, blocking, ducking, tackling, fooling opponents, kicking, and defending using both arms and legs. A highlight of the learned behaviors can be found here: this https URL

Subfields: Artificial Intelligence, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yoshua Bengio
  4. Mohit Bansal
  5. Ruslan Salakhutdinov
  6. Quoc V. Le
  7. Chelsea Finn
  8. Jianfeng Gao

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Shimon Whiteson
  5. Nicolas Manfred Otto Heess
  6. Yuxi Li
  7. Ofir Nachum
  8. Shixiang Shane Gu

In Game & Decision Theory

  1. Karl Tuyls
  2. Julien Pérolat
  3. Marc Lanctot
  4. Zhuoran Yang
  5. Thore Graepel
  6. Pablo Hernandez-Leal
  7. Matthew E. Taylor
  8. David Silver

Citations in the corpus