CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

By Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, Zhenhui Jessie Li. The Web Conference, 2019

372 citations recorded by the source corpus.

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

Subfields: Building and Construction, Control and Systems Engineering, Transportation

View Paper: https://www.semanticscholar.org/paper/86a1a83d5ffc3875a523cb8f6ce416b70c38e984

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

1. Hua Wei
2. Guanjie Zheng
3. Chacha Chen
4. Kai Xu
5. Nan Xu
6. Zhenhui Jessie Li
7. Vikash V. Gayah
8. Yuanhao Xiong

In Control and Systems Engineering

1. Hua Wei
2. Guanjie Zheng
3. Chacha Chen
4. Kai Xu
5. Nan Xu
6. Vikash V. Gayah
7. Zhenhui Jessie Li
8. Yuanhao Xiong

In Transportation

1. Yuanhao Xiong
2. Hua Wei
3. Chacha Chen
4. Yanmin Zhu
5. Nan Xu
6. Weinan Zhang
7. Zhenhui Jessie Li
8. Kai Xu

Citations in the corpus

• PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network (2019, methodology): “6 EXPERIMENT We conduct experiments on CityFlow4, an open-source traffic simulator that supports large-scale traffic signal control [33].”
• A review of cooperative multi-agent deep reinforcement learning (2019, background): “In the area of traffic management, [254] provided CityFlow, a new traffic-signal environment to be used for MARL researches.”
• CoLight: Learning Network-level Cooperation for Traffic Signal Control (2019, methodology): “We conduct experiments on CityFlow1, an open-source traffic simulator that supports large-scale traffic signal control [35].”
• Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control (2020, methodology): “Following the previous work on traffic signal control study (Wei et al. 2018), we conduct experiments on Cityflow (Zhang et al. 2019) 6 .”