Using a Deep Reinforcement Learning Agent for Traffic Signal Control

By Wade Genders, Saiedeh Navabzadeh Razavi. arXiv.org, 2016

321 citations recorded by the source corpus.

Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82%, average queue length by 66% and average travel time by 20%.

Subfields: Building and Construction, Control and Systems Engineering, Transportation

View Paper: https://www.semanticscholar.org/paper/5f5242d6e3e9fceb9568f9d2658a2d1aacb475c0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

1. Hua Wei
2. Guanjie Zheng
3. Zhenhui Jessie Li
4. Lara Codecà
5. Huaxiu Yao
6. Zhu Han
7. Guiling Wang
8. Jie Wang

In Control and Systems Engineering

1. Hua Wei
2. Guanjie Zheng
3. Zhenhui Jessie Li
4. Hongyuan Zha
5. Lara Codecà
6. Zhu Han
7. Guiling Wang
8. Jie Wang

In Transportation

1. Hongyuan Zha
2. Lara Codecà
3. Hua Wei
4. Zhenhui Jessie Li
5. Zhu Han
6. Guiling Wang
7. Jie Wang
8. Guanjie Zheng

Citations in the corpus

• IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control (2018, background): “As previousmethods [9, 10, 15, 22] store all the Research Track Paper KDD 2018, August 19‒23, 2018, London, United Kingdom”
• Integrating Independent and Centralized Multi-agent Reinforcement Learning for Traffic Signal Network Optimization (2019, background): “While early works proposed to optimize traffic light controllers based on expert knowledge and traditional model-based planning [4, 9, 18], there are promising recent results on applying flexible model-free methods in reinforcement learning (RL) [21] and deep RL, such as DQN in particular [16], to f”
• A Deep Reinforcement Learning Network for Traffic Light Cycle Control (2018, methodology): “When much useful relevant information is omitted in the limited states, it seems unable to act optimally in traffic light control [9].”
• Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning (2017, background): “Although a few previous studies have tried to apply deep reinforcement learning in the traffic signal control problem [38, 14], in this research the state representation is different.”
• Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control (2019, methodology): “[32] applied deep Q-learning in a more realistic traffic network, but with infeasible microscopic discrete traffic state.”