Model-free Deep Reinforcement Learning for Urban Autonomous Driving

By Jianyu Chen, Bodi Yuan, Masayoshi Tomizuka. 2019

318 citations recorded by the source corpus.

Urban autonomous driving decision making is challenging due to complex road geometry and multi-agent interactions. Current decision making methods are mostly manually designing the driving policy, which might result in suboptimal solutions and is expensive to develop, generalize and maintain at scale. On the other hand, with reinforcement learning (RL), a policy can be learned and improved automatically without any manual designs. However, current RL methods generally do not work well on complex urban scenarios. In this paper, we propose a framework to enable model-free deep reinforcement learning in challenging urban autonomous driving scenarios. We design a specific input representation and use visual encoding to capture the low-dimensional latent states. Several state-of-the-art model-free deep RL algorithms are implemented into our framework, with several tricks to improve their performance. We evaluate our method in a challenging roundabout task with dense surrounding vehicles in a high-definition driving simulator. The result shows that our method can solve the task well and is significantly better than the baseline.

Subfields: Artificial Intelligence, Automotive Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. W. B. Knox
2. Peter Stone
3. Jianyu Chen
4. X. Guan
5. Tao Jiang
6. Masayoshi Tomizuka
7. Shengbo Eben Li
8. Felix Schmitt

In Automotive Engineering

1. Jianyu Chen
2. Peter Stone
3. Masayoshi Tomizuka
4. Shengbo Eben Li
5. Pravendra Singh
6. Pranav Singh Chib
7. Xiaosong Hu

In Control and Systems Engineering

1. Peter Stone
2. W. B. Knox
3. Jianyu Chen
4. Masayoshi Tomizuka
5. Shengbo Eben Li
6. Felix Schmitt
7. Faraz Torabi
8. Garrett Warnell

Citations in the corpus

• Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning (2020, background): “Note this variant can be regarded as an extension of the article [46], which separately trains a non-sequential variational auto-encoder (VAE) to obtain the latent state, and then applies RL on the latent space.”
• A Review of Deep Reinforcement Learning for Smart Building Energy Management (2020, background): “, games [88] [31], autonomous driving [32]–[35], autonomous IoT [36], smart buildings [19] [20] [37] [38], smart city [39], wireless networks [40], Internet of energy [41], unmanned aerial vehicles [42], smart microgrids [43], edge computing [44], and manufacturing systems [45].”
• Reward (Mis)design for Autonomous Driving (2021, background): “3 of these 8 papers did report return [53,21,9].”