Mastering the game of Go without human knowledge

By David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, L. Baker, Matthew Lai, Adrian Bolton and 7 more. 2017

10,580 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In Reinforcement Learning

Citations in the corpus

Scaling Laws for Transfer (2021, background): “OpenAI Five played over 10,000 years worth of Dota, AlphaZero played 140 million games of Go, and GPT-3 read a signiﬁcant fraction of the internet [OBB + 19, SSS + 17, BMR + 20].”
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence (2020, background): “In one of DeepMind’s most influential paper “Mastering the game of Go without human knowledge”, the very goal was to dispense with human knowledge altogether, so as to “learn, tabula rasa, superhuman proficiency in challenging domains” (Silver et al., 2017).”
Are we done with ImageNet? (2020, background): “For example, progress on ImageNet due to architecture design [2, 3, 4] or optimization [5] has yielded corresponding gains on other tasks and modalities [6, 7, 8].”
Group Normalization (2018, background): “We will also investigate GN’s performance on learning representations for reinforcement learning (RL) tasks, e.g ., [54], where BN is playing an important role for training very deep models [20].”
Energy and Policy Considerations for Modern Deep Learning Research (2020, methodology): “…methodology and computational hardware have enabled exciting advances across many application areas of artiﬁcial intelligence, such as game playing (Silver et al. 2017; OpenAI 2018), natural language processing (Devlin et al. 2019; Aharoni, Johnson, and Firat 2019), computer vision (Chollet 2017;…”
Scalable agent alignment via reward modeling: a research direction (2018, background): “• Training a new model to predict answers to questions without having to expand the subquestions, akin to using a value network to the estimate the result of a tree search (Anthony et al., 2017; Silver et al., 2017).”
Supervising strong learners by amplifying weak experts (2018, methodology): “Expert Iteration: our method is very similar to Expert Iteration (ExIt) (Anthony et al., 2017) and AlphaZero (Silver et al., 2017b,a) and has recently achieved strong performance in the board games Hex, Go, Chess, and Shogi.”
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (2017, methodology): “Unless otherwise specified, the training and search algorithm and parameters are identical to AlphaGo Zero (31).”
Thinking Fast and Slow with Deep Learning and Tree Search (2017, methodology): “To do this, we play multiple games with the exploration policy, and select a single state from each game, as in Silver et al. [13].”
Neural Logic Machines (2019, background): “…applications such as speech recognition (Hinton et al., 2012), image classiﬁcation (Krizhevsky et al., 2012; He et al., 2016), machine translation (Sutskever et al., 2014; Bahdanau et al., 2015; Wu et al., 2016; Vaswani et al., 2017), and game playing (Mnih et al., 2015; Silver et al., 2017).”
Measuring the Algorithmic Efficiency of Neural Networks (2020, background): “It seems to us as if language modeling [Devlin et al., 2018, Radford et al., , Raffel et al., 2019] and playing games [Silver et al., 2016, Silver et al., 2017, Silver et al., 2018,OpenAI et al., 2019] are the domains of interest given our criteria.”
The jamming transition as a paradigm to understand the loss landscape of deep neural networks (2018, background): “[5] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al.”
A Convergence Theory for Deep Learning via Over-Parameterization (2018, background): “Neural networks have demonstrated a great success in numerous machine-learning tasks [6, 25, 30, 33, 36, 46, 47].”
Learning Latent Dynamics for Planning from Pixels (2018, background): “Planning is a natural and powerful approach to decision making problems with known dynamics, such as game playing and simulated robot control (Tassa et al., 2012; Silver et al., 2017; Morav ˇ cík et al., 2017).”
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures (2018, methodology): “Deep reinforcement learning methods have recently mastered a wide variety of domains through trial and error learning (Mnih et al., 2015; Silver et al., 2017; 2016; Zoph et al., 2017; Lillicrap et al., 2015; Barth-Maron et al., 2018).”
TreeQN and ATreeC: Differentiable Tree Planning for Deep Reinforcement Learning (2017, methodology): “The predictron (Silver et al., 2017b) instead learns abstract-state transition functions in order to predict values.”