Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control

By Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, Zhenhui. AAAI Conference on Artificial Intelligence, 2020

480 citations recorded by the source corpus.

Traffic congestion plagues cities around the world. Recent years have witnessed an unprecedented trend in applying reinforcement learning for traffic signal control. However, the primary challenge is to control and coordinate traffic lights in large-scale urban networks. No one has ever tested RL models on a network of more than a thousand traffic lights. In this paper, we tackle the problem of multi-intersection traffic signal control, especially for large-scale networks, based on RL techniques and transportation theories. This problem is quite difficult because there are challenges such as scalability, signal coordination, data feasibility, etc. To address these challenges, we (1) design our RL agents utilizing ‘pressure’ concept to achieve signal coordination in region-level; (2) show that implicit coordination could be achieved by individual control agents with well-crafted reward design thus reducing the dimensionality; and (3) conduct extensive experiments on multiple scenarios, including a real-world scenario with 2510 traffic lights in Manhattan, New York City 1 2.

Subfields: Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/794c17f1c7564ce6cc0d9d409aef73c8a6a79f59

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

A Survey on Traffic Signal Control Methods (2019, methodology): “. 2017, 2018b; Cahill et al. 2010; Salkham et al. 2008; Wei et al. 2018; Xu et al. 2013] Frequency of signal change [van der Pol 2016; Wei et al. 2018] Accident avoidance [van der Pol 2016] Pressure [Chen et al. 2020; Wei et al. 2019a] In the traffic signal control problem, although the ultimate obj”
Near-Optimal Model-Free Reinforcement Learning in Non-Stationary Episodic MDPs (2021, background): “…decision-making problems, such as online advertisement auctions (Cai et al., 2017; Lu et al., 2019), dynamic pricing (Chawla et al., 2016; Mao et al., 2018), trafﬁc management (Chen et al., 2020), health-care operations (Shortreed et al., 2011), and inventory con-trol (Agrawal & Jia, 2019).”
A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “To address these issues, [33] proposed MPLight, an RL algorithm for large-scale traffic signal control systems.”