Grandmaster level in StarCraft II using multi-agent reinforcement learning

By Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, A. Dudzik, Junyoung Chung, David Choi, Richard Powell, Timo Ewalds, Petko Georgiev and 32 more. 2019

4,437 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Clinical Psychology, Sociology and Political Science, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Clinical Psychology

In Sociology and Political Science

In Reinforcement Learning

Citations in the corpus

Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “This mitigation can be seen as analogous to Synnaeve et al. (2019) and Vinyals et al. (2019) in the RL literature.”
Competition-level code generation with AlphaCode (2022, methodology): “We believe that sampling metadata leads to a more diverse set of model samples, a strategy similar to that used by Vinyals et al. (2019), and allows our model to take advantage of the relative strengths across the metadata distribution.”
Augmented Language Models: a Survey (2023, methodology): “…feedback-based interaction with an environment, and it has been used for applications such as playing games (Mnih et al., 2015; Silver et al., 2016; Vinyals et al., 2019; Team et al., 2021; Bakhtin et al., 2022) or controlling robots (Gu et al., 2017; Kalashnikov et al., 2018; Akkaya et al., 2019;…”
Object-Centric Learning with Slot Attention (2020, background): “[5] Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al.”
Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, methodology): “[1, 89, 65]), and the shape and structure of a task’s inputs and outputs may place strong constraints on how such a system processes data, making it difficult to adapt to new settings.”
Imitating Interactive Intelligence (2020, methodology): “Imitation learning has been successfully deployed to build agents for self-driving cars (Pomerleau, 1989), robotics and biomimetic motor control (Schaal, 1999), game play (Silver et al., 2016; Vinyals et al., 2019), and language modeling (Shannon, 1951).”
Generative Language Modeling for Automated Theorem Proving (2020, background): “Artiﬁcial neural networks have enjoyed a spectacularly successful decade, having made considerable advances in computer vision [1, 2], translation [3, 4, 5], speech recognition [6, 7], image generation [8, 9, 10, 11, 12], game playing [13, 14, 15], and robotics [16, 17].”
Toward Causal Representation Learning (2021, background): “Machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efficient and generalize better [9], [11], [15], [16], [27], [58], [77], [84], [85], [141], [157], [177], [181], [197], [211], [212], [244], [258], [272], [274].”
The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (2021, background): “Recent advances in reinforcement learning (RL) and multi-agent reinforcement learning (MARL) have led to a great deal of progress in creating artificial agents which can cooperate to solve tasks: DeepMind’s AlphaStar surpassed professional-level performance in the StarCraft II [34], OpenAI Five defe”
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, methodology): “The full game of StarCraft II has already been used as an RL environment (Vinyals et al., 2017), and DeepMind’s AlphaStar (Vinyals et al., 2019) has recently shown an impressive level of play on a StarCraft II matchup using a centralised controller.”
Towards Causal Representation Learning (2021, background): “accumulating evidence supporting the usefulness of an appropriate structured representation of the environment [2, 26, 258].”
Dota 2 with Large Scale Deep Reinforcement Learning (2019, background): “The model for AlphaStar used a similar hand-designed architecture to embed observations and an autoregressive action decoder, with an LSTM core to handle partial observability.”
Transfer without Forgetting (2022, background): “Thanks to the enthusiastic development carried out by the scientific community, there exist myriad widely available deep learning models that can be either readily deployed or easily adapted to perform complex tasks [24,60,64,48,4].”
MOReL : Model-Based Offline Reinforcement Learning (2020, background): “Unfortunately, a direct embodiment of this trial and error learning is often inefficient and feasible only with a simulator [9, 10, 11].”
Regularized Behavior Value Estimation (2021, methodology): “This paradigm has attained considerable success on Atari (Mnih et al., 2015), Go (Silver et al., 2017), StarCraft II and Dota 2 (Vinyals et al., 2019; Berner et al., 2019), and robotics (Andrychowicz et al., 2020).”
Benchmarks for Deep Off-Policy Evaluation (2021, background): “…learning algorithms can acquire effective policies for a wide range of problems through active online interaction, such as in robotics (Kober et al., 2013), board games and video games (Tesauro, 1995; Mnih et al., 2013; Vinyals et al., 2019), and recommender systems (Aggarwal et al., 2016).”
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, background): “One potential approach to enable a robot to perform these manipulation tasks is with deep reinforcement learning (RL), which has achieved many successes in recent years [4, 5, 6, 7, 8].”