Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

By Seyed Sajad Mousavi, Michael Schukat, Peter Corcoran, Enda Howley. arXiv.org, 2017

348 citations recorded by the source corpus.

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process.

Subfields: Automotive Engineering, Building and Construction, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/41aa579df5f20230404714e0d9902de927113ac9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Building and Construction

In Control and Systems Engineering

Citations in the corpus

SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach (2019, background): “In recent years, deep neural networks have shown impressive results in various domains ranging from computer vision and reinforcement learning to natural language processing [13], [14], [15], [16], [17], [18], [19].”
A Deep Reinforcement Learning Network for Traffic Light Cycle Control (2018, background): “Even worse, it may also incur vehicular accidents [1], [2].”
Integrating Independent and Centralized Multi-agent Reinforcement Learning for Traffic Signal Network Optimization (2019, background): “The success of deep RL has spurred recent works that incorporate high dimensional state information into a more realistic problem definition [10, 21, 31, 15], which we further extend to define the observation and action spaces of our new multi-agent setting.”
PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network (2019, background): “Recent work using deep RL [7, 14, 20, 25, 31, 34] can cope with more complex continuous state representation.”
A Survey on Traffic Signal Control Methods (2019, methodology): “011] Congestion [Bakker et al. 2010; Iša et al. 2006; Steingrover et al. 2005] Position of vehicles [Bakker et al. 2010; Iša et al. 2006; Khamis and Gomaa 2012; Khamis et al. 2012; Kuyer et al. 2008; Mousavi et al. 2017; Steingrover et al. 2005; van der Pol et al. 2016; Wei et al. 2018; Wiering 2000”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, background): “various problems such as controlling a group of autonomous vehicles or drones [123], robot soccer [124], controlling traffic signals [125], coordinating collaborative bots in factories and”
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey (2020, methodology): “The authors in [28], presented two deep RL algorithms for controlling isolated intersections: value-based DQN, and policy-based actor critic.”
Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset (2020, background): “, classification, real time prediction, and image segmentation) even more successful in different applications [23, 24, 25, 26].”