Deep Reinforcement Learning: An Overview

By Yuxi Li. 2017

1,862 citations recorded by the source corpus.

We give an overview of recent exciting achievements of deep reinforcement learning (RL). We discuss six core elements, six important mechanisms, and twelve applications. We start with background of machine learning, deep learning and reinforcement learning. Next we discuss core RL elements, including value function, in particular, Deep Q-Network (DQN), policy, reward, model, planning, and exploration. After that, we discuss important mechanisms for RL, including attention and memory, unsupervised learning, transfer learning, multi-agent RL, hierarchical RL, and learning to learn. Then we discuss various applications of RL, including games, in particular, AlphaGo, robotics, natural language processing, including dialogue systems, machine translation, and text generation, computer vision, neural architecture design, business management, finance, healthcare, Industry 4.0, smart grid, intelligent transportation systems, and computer systems. We mention topics not reviewed yet, and list a collection of RL resources. After presenting a brief summary, we close with discussions. Please see Deep Reinforcement Learning, arXiv:1810.06339, for a significant update.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “Targeting the same problem as in Reference [184], Jagannatha et al. [92] analyzed the properties and limitations of standard off-policy evaluation methods in RL and discussed possible extensions to improve their utility in clinical domains.”
Continuous Doubly Constrained Batch Reinforcement Learning (2021, background): “Barring a few examples [37], advancements have not translated quite as well to real-world environments, where it is typically infeasible to experience millions of environmental interactions [11].”
A Brief Survey of Deep Reinforcement Learning (2017, neutral): “language processing [106, 5], we refer readers to the overview by Li [78].”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “fake” generated samples, while the second (generator) network is trained to generate samples that are as similar to the real dataset as possible. The class of Reinforcement Learning (RL) utilizes DNNs[154] for different purposes, such as defining policy functions and reward functions. Training algor”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “.2we outlined some recent works in single-agent DRL since an exhaustive list is out of the scope of this article. This explosion of works has led DRL to be extended and combined with other techniques [23, 37, 29]. One natural extension to DRL is to test whether these approaches could be applied in a”
Knowledge-Guided Agent-Tactic-Aware Learning for StarCraft Micromanagement (2018, background): “…(DRL) [Mnih et al. , 2015; Li, 2017; Silver et al. , 2016; Lillicrap et al. , 2015; Levine et al. , 2016; Mirowski et al. , 2017; Pasunuru and Bansal, 2017; Li et al. , 2017; Silver et al. , 2017] has at-tracted considerable attention in the ﬁelds of artiﬁcial intelligence and machine learning.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, methodology): “DRL is also applicable and shows potentials of application in healthcare, finance and business management [13, 105], where human security and property safety issues should be considered, leading to the demands of explainable RL [134].”
CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario (2019, background): “At the same time, traffic signal control is also one of the major real-world application scenarios for reinforcement learning [6].”
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues (2018, background): “Recently, some enhancements to DRL have come out, and readers can refer to the literature [38] for more details related to DRL .”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “We discuss two typical reinforcement models: Deep Q-Networks (Mnih et al., 2015) and REINFORCE (Williams, 1992; Sutton et al., 1999).”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “discuss the applications of DRL such as [7] and [8], but they are specifically for computer vision and natural language processing.”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, background): “1 Deep Q-Network Deep RL is a broad term that indicates the combination between deep learning and RL to deal with highdimensional environments [3, 57, 69].”
A Review of Deep Reinforcement Learning for Smart Building Energy Management (2020, background): “As a general artificial intelligence technology, deep reinforcement learning (DRL) [28] [29] is promising to address the above challenges and has been applied in many fields, e.”
A survey on large language model based autonomous agents (2023, unknown): “In the field of reinforcement learning, agents learn by continuously exploring the environment and adapting based on environmental feedback [68, 82, 98, 152].”
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches (2018, methodology): “There is good survey which is based on Reinforcement Learning approaches [40, 41].”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “For a deeper exploration of RL methods, readers can refer to more comprehensive reviews provided by [289], [290], and [291]. a) Deep reinforcement learning.”