End-to-End Deep Reinforcement Learning for Lane Keeping Assist

By Ahmad El Sallab, Mohammed Abdou, Etienne Perot, Senthil Kumar Yogamani. 2016

184 citations recorded by the source corpus.

Reinforcement learning is considered to be a strong AI paradigm which can be used to teach machines through interaction with the environment and learning from their mistakes, but it has not yet been successfully used for automotive applications. There has recently been a revival of interest in the topic, however, driven by the ability of deep learning algorithms to learn good representations of the environment. Motivated by Google DeepMind's successful demonstrations of learning for games from Breakout to Go, we will propose different methods for autonomous driving using deep reinforcement learning. This is of particular interest as it is difficult to pose autonomous driving as a supervised learning problem as it has a strong interaction with the environment including other vehicles, pedestrians and roadworks. As this is a relatively new area of research for autonomous driving, we will formulate two main categories of algorithms: 1) Discrete actions category, and 2) Continuous actions category. For the discrete actions category, we will deal with Deep Q-Network Algorithm (DQN) while for the continuous actions category, we will deal with Deep Deterministic Actor Critic Algorithm (DDAC). In addition to that, We will also discover the performance of these two categories on an open source car simulator for Racing called (TORCS) which stands for The Open Racing car Simulator. Our simulation results demonstrate learning of autonomous maneuvering in a scenario of complex road curvatures and simple interaction with other vehicles. Finally, we explain the effect of some restricted conditions, put on the car during the learning phase, on the convergence time for finishing its learning phase.

Subfields: Artificial Intelligence, Automotive Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

Citations in the corpus

A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, background): “The lane keeping performance of (a) the DQN with discretised outputs and (b) DDAC with continuous output values [122].”
Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge (2017, background): “For instance, Sallab et al. [11] used end-to-end deep reinforcement learning for lane-keeping assist on an open-source simulator for Racing called (TORCS).”
Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles (2020, background): “position and angle of the vehicle in lane [78], [60], [22], while the second is the image of a front-facing camera [54], [59], [57].”
Exploring applications of deep reinforcement learning for real-world autonomous driving systems (2019, methodology): “In (Sallab et al., 2016) DRL system for lane keeping assist is introduced for discrete actions (DQN) and continuous actions (DDAC), where the TORCS car simulator is used and concluded that, as expected, the continuous actions provide smoother trajectories, and the more restricted termination…”
Synthetic Data for Deep Learning (2019, methodology): “[516,517] use it in their deep reinforcement learning frameworks for lane keeping assist and autonomous driving, Xiong et al.”
A Survey of State-Action Representations for Autonomous Driving (2018, methodology): “As noted by Sallab et al. (2016) who compares the continuous and discrete settings, the discretization implies the introduction of discontinuities in the commands, which can lead to instability and jerky trajectories.”
A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers (2018, methodology): “Recently, it has been used in automated vehicle field [12, 13].”
Urban Driving with Multi-Objective Deep Reinforcement Learning (2018, methodology): “Most existing RL approaches for autonomous driving consider a state space of either raw visual/sensor input [8, 20], or the kinematics of a few immediately surrounding vehicles [14, 27].”
Virtual to Real Reinforcement Learning for Autonomous Driving (2017, background): “Recently, reinforcement learning has been considered as a promising technique to learn driving policy due to its expertise in action planing [15, 23, 25].”