Mastering the game of Go with deep neural networks and tree search

By David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot and 10 more. 2016

19,057 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Developmental and Educational Psychology, Economics and Econometrics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Developmental and Educational Psychology

In Economics and Econometrics

In Reinforcement Learning

Citations in the corpus

Scaling Laws for Transfer (2021, background): “…important to characterize ﬁne-tuning in the low data regime because many tasks of interest won’t have a sufﬁciently big, readily available dataset to train large models from-scratch (either billions of data points [HNA + 17, KMH + 20, HKK + 20] or a perfect simulator like in Go [SHM + 16]).”
Augmented Language Models: a Survey (2023, methodology): “…behaviors through feedback-based interaction with an environment, and it has been used for applications such as playing games (Mnih et al., 2015; Silver et al., 2016; Vinyals et al., 2019; Team et al., 2021; Bakhtin et al., 2022) or controlling robots (Gu et al., 2017; Kalashnikov et al., 2018;…”
Deep Reinforcement Learning from Human Preferences (2017, background): “Recent success in scaling reinforcement learning (RL) to large problems has been driven in domains that have a well-speciﬁed reward function (Mnih et al., 2015, 2016; Silver et al., 2016).”
mixup: Beyond Empirical Risk Minimization (2017, background): “Large deep neural networks have enabled breakthroughs in ﬁelds such as computer vision (Krizhevsky et al., 2012), speech recognition (Hinton et al., 2012), and reinforcement learning (Silver et al., 2016).”
Emergence of Locomotion Behaviours in Rich Environments (2017, background): “Reinforcement learning has demonstrated remarkable progress, achieving high levels of performance in Atari games [1], 3D navigation tasks [2, 3], and board games [4].”
Sample Efficient Actor-Critic with Experience Replay (2016, background): “Policy gradient methods have been at the heart of significant advances in AI and robotics (Silver et al., 2014; Lillicrap et al., 2015; Silver et al., 2016; Levine et al., 2015; Mnih et al., 2016; Schulman et al., 2015a; Heess et al., 2015).”
One-Shot Imitation Learning (2017, background): “…learning has had many successes, including Backgammon (Tesauro, 1995), helicopter control (Ng et al., 2003), Atari (Mnih et al., 2015), Go (Silver et al., 2016), continuous control in simulation (Schulman et al., 2015; Heess et al., 2015; Lillicrap et al., 2015) and on real robots (Peters…”
Scalable agent alignment via reward modeling: a research direction (2018, background): “…stunt their performance: the choice of algorithms—e.g. A3C (Mnih et al., 2016) vs. IMPALA (Espeholt et al., 2018)—the number of training steps, the choice of training environments, the model capacity, the planning horizon, the number of Monte Carlo tree search rollouts (Silver et al., 2016), etc.”
Reward learning from human preferences and demonstrations in Atari (2018, background): “Finally, the first published version of AlphaGo (Silver et al., 2016) pretrains from human demonstrations.”
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (2017, methodology): “Recently, the AlphaGo Zero algorithm achieved superhuman performance in the game of Go, by representing Go knowledge using deep convolutional neural networks (23, 30), trained solely by reinforcement learning from”
Constrained Policy Optimization (2017, background): “…performance on many high-dimensional control tasks, including Atari games (using pixels as inputs) (Mnih et al., 2015; 2016), robot locomotion and manipulation (Schulman et al., 2015; Levine et al., 2016; Lillicrap et al., 2016), and even Go at the human grandmaster level (Silver et al., 2016).”
Thinking Fast and Slow with Deep Learning and Tree Search (2017, methodology): “[12], which achieved state-of-the-art performance for a neural network player in the related board game Go.”
AI safety via debate (2018, methodology): “For example, the ﬁrst AlphaGo result [Silver et al., 2016] used an opponent pool but later versions do not [Silver et al., 2017a,b].”
Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning (2018, background): “Recent work has received high attention by successfully scaling reinforcement learning (RL) to games with large state-action spaces, achieving human-level (Mnih et al., 2015) or even superhuman performance (Silver et al., 2016).”
Imitating Interactive Intelligence (2020, methodology): “Imitation learning has been successfully deployed to build agents for self-driving cars (Pomerleau, 1989), robotics and biomimetic motor control (Schaal, 1999), game play (Silver et al., 2016; Vinyals et al., 2019), and language modeling (Shannon, 1951).”