Deep Reinforcement Learning framework for Autonomous Driving
By Ahmad El Sallab, Mohammed Abdou, Etienne Perot, Senthil Kumar Yogamani. 2017
1,062 citations recorded by the source corpus.
Reinforcement learning is considered to be a strong AI paradigm which can be used to teach machines through interaction with the environment and learning from their mistakes. Despite its perceived utility, it has not yet been successfully applied in automotive applications. Motivated by the successful demonstrations of learning of Atari games and Go by Google DeepMind, we propose a framework for autonomous driving using deep reinforcement learning. This is of particular relevance as it is difficult to pose autonomous driving as a supervised learning problem due to strong interactions with the environment including other vehicles, pedestrians and roadworks. As it is a relatively new area of research for autonomous driving, we provide a short overview of deep reinforcement learning and then describe our proposed framework. It incorporates Recurrent Neural Networks for information integration, enabling the car to handle partially observable scenarios. It also integrates the recent work on attention models to focus on relevant information, thereby reducing the computational complexity for deployment on embedded hardware. The framework was tested in an open source 3D car racing simulator called TORCS. Our simulation results demonstrate learning of autonomous maneuvering in a scenario of complex road curvatures and simple interaction of other vehicles.
Subfields: Artificial Intelligence, Automotive Engineering, Reinforcement Learning, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Ruslan Salakhutdinov
- Pieter Abbeel
- Dacheng Tao
- Mohit Bansal
- Percy Liang
- Quoc V. Le
In Automotive Engineering
- Sergey Levine
- Aviral Kumar
- Jacob Lambert
- Alexander Carballo
- Ekim Yurtsever
- Aravind Rajeswaran
- Sergey I. Nikolenko
- George Tucker
In Reinforcement Learning
- Sergey Levine
- Aravind Rajeswaran
- Pieter Abbeel
- Aviral Kumar
- Chelsea Finn
- Ofir Nachum
- George Tucker
- Shixiang Shane Gu
In Robotics & Control
- Sergey Levine
- Aravind Rajeswaran
- George Tucker
- Justin Fu
- Sergey I. Nikolenko
- Pieter Abbeel
- Aviral Kumar
- Chelsea Finn
Citations in the corpus
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “Reinforcement learning approaches have been applied in both simulation (Sallab et al., 2017) and in the real world, with human interventions in case the vehicle violates a safety constraint (Kendall et al., 2019) (see Figure 7).”
- Self-Driving Car Steering Angle Prediction Based on Image Recognition (2019, methodology): “Inspired by these work, [3] has proposed a framework for autonomous driving using deep RL.”
- Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Learn policies to automatically park the vehicle [90].”
- MOReL : Model-Based Offline Reinforcement Learning (2020, background): “[12], and has applications in healthcare [33, 34, 35], recommendation systems [36, 37, 38, 39], dialogue systems [40, 19, 41], and autonomous driving [42].”
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “introduced in [60], where the network was tested in a simulation environment.”
- Safe Exploration in Continuous Action Spaces (2018, background): “However, RL is recently also finding its path into industrial applications in the physical world; e.g., datacenter cooling (Evans & Gao, 2016), robotics (Gu et al., 2016), and autonomous vehicles (Sallab et al., 2017).”
- Reinforcement Learning based Recommender Systems: A Survey (2021, background): “This made it possible to apply RL in problems with enormous state and action spaces, including self-driving cars [9, 10], robotics [11], industry automation [12], finance [13], healthcare [14, 15], and RSs.”
- Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving (2020, background): “Deep Reinforcement Learning (DRL) based automated driving agents [20], [21] replaced the need for huge amounts of labeled data with online interaction.”
- A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “transportation schedule [22, 53, 159, 253], traffic signal control [50, 59], energy management [180], wireless security [172], satellite docking [76], edge computing [299], chemical processes [241], video games [28, 153, 192, 211, 262, 300], board games of Go, shogi, chess and arcade game PAC-MAN [1”
- Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges (2021, background): “such as playing games [4], robotics [5], autonomous driving [6], clinical trials [7], etc.”
- A Survey of State-Action Representations for Autonomous Driving (2018, neutral): “This representation is used by (Ziebart et al., 2009; Richter et al., 2014; Chen et al., 2016; Tamar et al., 2016; Shankar et al., 2016; Wulfmeier et al., 2016; Sallab et al., 2017; Rehder et al., 2017a; Hoermann et al., 2017; Williams et al., 2017; Mukadam et al., 2018; Rhinehart et al., 2018).”
- Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation (2019, background): “There has been impressive progress on deep reinforcement learning (RL) for many tasks, such as Atari video games, GO [26, 35], robot control [10], self-driving [33] and navigation [40].”
- Distributed Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dynamic Coordination Graphs (2020, background): “end-to-end learning of driving actions in partially observable scenarios [15] or from raw visual inputs [39].”
- Exploring applications of deep reinforcement learning for real-world autonomous driving systems (2019, neutral): “Frameworks A framework for an end-end Deep Re-inforcement Learning pipeline for autonomous driving is proposed in (Sallab et al., 2017), where the inputs are the states of the environment and their ag-gregations over time, and the output is the driving actions.”
- LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning (2019, background): “Many real-world problems, such as traffic light control [1], coordination of autonomous vehicles [2], resources management [3] and multi-player video games [4, 5], can be naturally formulated into cooperative multi-agent systems, where the objective is to maximize the return in the perspective of a ”
- Continuous control for robot based on deep reinforcement learning (2019, methodology): “[50] used an end-to-end method via DQN to learn the driving behaviors, which received the raw sensor data and outputs the driving actions.”
- A survey of deep learning techniques for autonomous driving (2019, methodology): “In Sallab et al. (2017b), a DRL End2End system is used to compute steering command in the TORCS game simulation engine.”