A Brief Survey of Deep Reinforcement Learning
By Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, Anil Anthony Bharath. 2017
3,581 citations recorded by the source corpus.
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higherlevel understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Dacheng Tao
- Mohit Bansal
- Quoc V. Le
- Jianfeng Gao
- Percy Liang
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Jitendra Malik
- Xiaogang Wang
- Cordelia Schmid
- Bernt Schiele
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Silvio Savarese
- David R Ha
- Chelsea Finn
- Roberto Mart'in-Mart'in
- Yuke Zhu
- Ajay Mandlekar
Citations in the corpus
- DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “Large deep neural network have resulted in breakthroughs across computer vision [1], speech recognition [2] and reinforcement learning [3] with their success largely attributed to their ability to efficiently learn complex patterns from data.”
- World Models (2018, background): “Although there is a large body of research relating to model-based reinforcement learning, this article is not meant to be a review (Arulkumaran et al., 2017; Schmidhuber, 2015b) of the current state of the field.”
- Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation (2020, background): “Deep reinforcement learning (DRL) algorithms have achieved many successes in solving problems with high-dimensional state and action spaces [27, 3].”
- Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy (2019, background): “…specifically at the field of Reinforcement Learning (RL), which in recent years has demonstrated impressive results on tasks such as the games of Go and Dota2, walking as a simulated human-like agent, and accomplishing a variety of real-world robotic tasks [Gitau, 2019; Arulkumaran et al. , 2017].”
- Continual Reinforcement Learning in 3D Non-stationary Environments (2019, background): “In the last decade we have witnessed a renewed interest and major progresses in reinforcement learning (RL) especially due to recent deep learning developments [3].”
- IQA: Visual Question Answering in Interactive Environments (2017, neutral): “For more comprehensive surveys of pre-deep and post-deep era applications of RL techniques, we refer the reader to [4, 28].”
- Revisiting Small Batch Training for Deep Neural Networks (2018, background): “…enabled significant advances in a number of applications, including computer vision, speech recognition and natural language processing, and in reinforcement learning for robotic control and game playing (LeCun et al., 2015; Schmidhuber, 2015; Goodfellow et al., 2016; Arulkumaran et al., 2017).”
- Video-to-Video Synthesis (2018, background): “For example, in model-based reinforcement learning [2,24], a video synthesis model finds use in approximating visual dynamics of the world for training the agent with less amount of real experience data.”
- A survey and critique of multiagent deep reinforcement learning (2018, neutral): “We refer the reader to recent surveys on single-agent DRL [13,101,191] for a more detailed discussion of the literature.”
- Fusing Blockchain and AI With Metaverse: A Survey (2022, methodology): “These DRL algorithms can be applied to achieve over-human performance in some fields [44].”
- Reinforcement learning for control: Performance, stability, and deep approximators (2018, neutral): “For a broader view of DRL we refer the reader to the recent reviews (Arulkumaran et al., 2017; Li, 2017).”
- Deep Reinforcement Learning for Cyber Security (2019, background): “With the power of function approximation and representation learning, deep learning can learn a compact low-dimensional representation of raw high-dimensional data [58].”
- Toward Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions (2018, methodology): “…rewards from the environment, resulting in the return of L l =0 γ l r l +1 , where γ ∈ [0 1] is the discount factor which provides more weights on the immediate rewards and L denotes the length of one episode, which denotes the time period after which the state is reset for an episodic MDP [45].”
- Voyager: An Open-Ended Embodied Agent with Large Language Models (2023, background): “Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8–10] that operate on primitive actions, which could be challenging for systematic exploration [11–15], interpretability [16–18], and generalization [19–21].”
- Communication-Efficient Edge AI: Algorithms and Systems (2020, background): “…has achieved breakthroughs in a wide range of applications, including speech processing [1], image classi-fication [2] and reinforcement learning [3], etc. AI is expected to affect significant segments of many vertical industries and our daily life, such as intelligent vehicles [4] and tactile…”
- A Survey of Machine Learning Techniques Applied to Software Defined Networking (SDN): Research Issues and Challenges (2019, neutral): “DRL [128]–[130].”