CARLA: An Open Urban Driving Simulator
By Alexey Dosovitskiy, Germán Ros, Felipe Codevilla, Antonio M. López, Vladlen Koltun. 2017
6,955 citations recorded by the source corpus.
We introduce CARLA, an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites and environmental conditions. We use CARLA to study the performance of three approaches to autonomous driving: a classic modular pipeline, an end-to-end model trained via imitation learning, and an end-to-end model trained via reinforcement learning. The approaches are evaluated in controlled scenarios of increasing difficulty, and their performance is examined via metrics provided by CARLA, illustrating the platform's utility for autonomous driving research. The supplementary video can be viewed at this https URL
Subfields: Artificial Intelligence, Automotive Engineering, Control and Systems Engineering, Evaluation & Benchmarks, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Mohit Bansal
- Yoshua Bengio
- Trevor Darrell
- Jianfeng Gao
- Quoc V. Le
- Ruslan Salakhutdinov
- Percy Liang
In Automotive Engineering
- Andreas Geiger
- Sergey Levine
- Kashyap Chitta
- Pravendra Singh
- Pranav Singh Chib
- Raquel Urtasun
- Sergey I. Nikolenko
- Philipp Krahenbuhl
In Control and Systems Engineering
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Abhinav Kumar Gupta
- Silvio Savarese
- Kristen Grauman
- Jitendra Malik
- Yuke Zhu
In Evaluation & Benchmarks
- Jitendra Malik
- Abhinav Kumar Gupta
- Vladlen Koltun
- Luc Van Gool
- Antonio Torralba
- Kristen Grauman
- Trevor Darrell
- Sanja Fidler
In Robotics & Control
- Sergey Levine
- Abhinav Kumar Gupta
- Pieter Abbeel
- Silvio Savarese
- Chelsea Finn
- Dhruv Batra
- Kristen Grauman
- Vladlen Koltun
Citations in the corpus
- Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, background): “No existing synthetic dataset or simulator addresses all of these limitations, including those that target outdoor scene understanding [8, 20, 23, 34, 38, 41, 55, 56, 58, 60, 65, 76, 77].”
- WoodScape: A Multi-Task, Multi-Camera Fisheye Dataset for Autonomous Driving (2019, methodology): “There are several popular synthetic datasets like SYNTHIA [44] and CARLA [13].”
- Learning To Simulate (2018, methodology): “We focus on traffic scenes as simulators/games for this scenario are publicly available (CARLA (Dosovitskiy et al., 2017) with Unreal engine (Epic-Games, 2018) as backend).”
- Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, methodology): “…criteria which have been integrated into reward functions when training DRL agents for AD tasks include: distance travelled towards a destination [79], speed of the ego vehicle [79]–[83], keeping the ego vehicle at a standstill [82], collisions with other road users or scene objects [79], [80],…”
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “Modalities Framerate Target Use Case Gazebo (Koenig et al., 2004) Sensor Plugins 10s+FPS General Purposes Vrep (Rohmer et al., 2013) Sensor Plugins 10s+FPS General Purposes Airsim (Shah et al., 2017) Depth/Color/Semantics 20s+FPS Autonomous Driving Carla (Dosovitskiy et al., 2017) Depth/Color/Semant”
- Driving Policy Transfer via Modularity and Abstraction (2018, methodology): “he first waypoint. 5 4 System Setup We evaluate the presented approach in simulation and on a physical vehicle (a 1/5-scale truck). Simulation. We use CARLA, an open-source simulator for urban driving [12]. The simulator provides access to sensor data from the ego-vehicle, as well as detailed privile”
- D4RL: Datasets for Deep Data-Driven Reinforcement Learning (2020, background): “(Non-RL policies, undirected and multitask data, realism) CARLA [4] is a high-fidelity autonomous driving simulator, where the agent controls the throttle (gas pedal), the steering, and the break pedal for the car, and receives 48x48 RGB images from the driver’s perspective as observations.”
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “14: A depth image produced from synthetic lidar data, generated in the CARLA [264] simulator. built with random tree-like branches.”
- Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, methodology): “We used the robot car simulator CARLA [39], a simulator with realistic renderings and physics.”
- Deep Imitative Models for Flexible Inference, Planning, and Control (2018, methodology): “In our experiments, our approach substantially outperforms both MBRL and IL: it can efficiently learn near-perfect driving through the static-world CARLA simulator from just 7,000 trajectories obtained from 19 hours of driving.”
- End-to-End Driving Via Conditional Imitation Learning (2017, methodology): “We use CARLA [10], an urban driving simulator, to corroborate design decisions and evaluate the proposed approach in a dynamic urban environment with traffic.”
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning (2020, background): “In recent years, advances in physics-based simulations and graphics have led to a series of simulated platforms and toolkits [1, 14, 8, 2, 16] that have accelerated scientific progress on robotics and embodied AI.”
- Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, background): “Similarly, there exists a number of robotics simulation benchmarks including visual navigation [39, 40, 41, 42, 43], autonomous driving [44, 45, 46], grasping [47, 48, 49], single-task manipulation [50], among others.”
- Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, methodology): “In driving, several simulators like CARLA [27] and AirSim [28] promise easier self driving research.”
- AI2-THOR: An Interactive 3D Environment for Visual AI (2017, background): “SYNTHIA [16], Virtual KITTI [9], TORCS [21], and CARLA [7] provide synthetic data for autonomous driving.”
- MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments (2017, methodology): “, 2014) and CARLA (Dosovitskiy et al., 2017) simulators have been used to study autonomous driving policies.”
- On Evaluation of Embodied Navigation Agents (2018, methodology): “Other simulators that have been used to study navigation include ViZDoom and DeepMind Lab [15, 2] (relatively uncluttered corridor layouts), and CARLA [10] (out-door urban environments).”
- nuScenes: A Multimodal Dataset for Autonomous Driving (2019, background): “tes that no information is provided. *[33] provides static depth maps. Moving objects in the scene (e.g. cars) are not present in those depth maps. yWe report numbers for annotated scenes only. CARLA [17], SYNTHIA [50], and Virtual KITTI [24] simulate virtual cities using game engines. Playing for B”
- VirtualHome: Simulating Household Activities Via Programs (2018, methodology): “Simulations using game engines have recently been developed to facilitate training visual models for autonomous driving [7, 19, 6], quadcopter flying [21], or other robotic tasks [5].”