An Introduction to Deep Reinforcement Learning

By Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare, Joelle Pineau. Found. Trends Mach. Learn., 2018

1,498 citations recorded by the source corpus.

Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decision-making tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the aspects related to generalization and how deep RL can be used for practical applications. We assume the reader is familiar with basic machine learning concepts.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/4b61c25a86083c20730c9b12737ac6ac4178c364

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Management Science and Operations Research

Citations in the corpus

Never Give Up: Learning Directed Exploration Strategies (2020, background): “ematurely to suboptimal policies, and never gather the data they need to learn. Ensuring that all state-action pairs are encountered inﬁnitely often is the general problem of maintaining exploration (François-Lavet et al., 2018; Sutton & Barto, 2018). The simplest approach for tackling this prob”
Boosted Curriculum Reinforcement Learning (2022, background): “The combination of reinforcement learning (RL) (Sutton & Barto, 2018) algorithms with powerful function approximators, i.e., deep RL (Franc¸ois-Lavet et al., 2018; Mnih et al., 2015), is a break-through towards solving complex decision making and control problems that were impractical for previous…”
Free-Lunch Saliency via Attention in Atari Agents (2019, background): “Over the last decade, reinforcement learning has shifted towards deep reinforcement learning, where policies and/or models of the environment are modeled with deep neural networks, with impressive results achieved across a wide variety of tasks, from game playing to robotics [2, 14].”
A review on Deep Reinforcement Learning for Fluid Mechanics (2019, background): “Going back to the end of the 1980s (Sutton, 1988), reinforcement learning (RL) proposed a formal framework in which an agent learns by interacting with an environment through the gathering of experience (François-lavet et al. , 2018).”
Recent advances in reinforcement learning in finance (2021, methodology): “We refer the reader to François-Lavet et al. (2018) for other deep value-based RL algorithms such as Neural TD learning and Dueling 𝑄-Networks.”
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, methodology): “Deep reinforcement learning (DRL) [147] techniques, such as the Markov decision process (MDP) [148], have recently been used to reformulate path-ﬁnding between entities as a sequential decision-making task.”
Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction (2019, neutral): “f NSD, and 0.61 according to the average reward. 1 Introduction Reinforcement learning has become increasingly popular due to its success in addressing challenging sequential decision-making problems [9]. Signiﬁcant progress has been made in many real-world applications, such as playing video games ”
Machine Learning in Aerodynamic Shape Optimization (2022, methodology): “Based on the way to obtain the optimal RL policy, there are two categories of DRL: value-based methods and policy-based methods [228, 230, 231].”
A Perspective on Objects and Systematic Generalization in Model-Based RL (2019, background): “In contrast, agents trained with Reinforcement Learning (RL) frequently fall short in this regard: they rely on excessive amounts of data (Franc¸ois-Lavet et al., 2018) and are unable to generalize beyond their initial training regime (Leike et al., 2017).”
Revisiting Fundamentals of Experience Replay (2020, background): “This problem is well studied, and a wide range of methods based on value iteration, policy iteration, and policy gradients have been built up over the course of decades (Bellman, 1957; Puterman, 1994; Bertsekas & Tsitsiklis, 1996; Kaelbling et al., 1996; Szepesvári, 2010; Sutton & Barto, 2018; Frano”
Data-Efficient Reinforcement Learning with Self-Predictive Representations (2020, methodology): “Moreover, to address ﬁndings from Henderson et al. (2018) that comparisons based on small numbers of random seeds are unreliable, we average our results over ten random seeds, twice as many as most previous works.”