DDD17: End-To-End DAVIS Driving Dataset

By Jonathan Binas, Daniel Neil, Shih-Chii Liu, Tobi Delbruck. 2017

242 citations recorded by the source corpus.

Event cameras, such as dynamic vision sensors (DVS), and dynamic and active-pixel vision sensors (DAVIS) can supplement other autonomous driving sensors by providing a concurrent stream of standard active pixel sensor (APS) images and DVS temporal contrast events. The APS stream is a sequence of standard grayscale global-shutter image sensor frames. The DVS events represent brightness changes occurring at a particular moment, with a jitter of about a millisecond under most lighting conditions. They have a dynamic range of >120 dB and effective frame rates >1 kHz at data rates comparable to 30 fps (frames/second) image sensors. To overcome some of the limitations of current image acquisition technology, we investigate in this work the use of the combined DVS and APS streams in end-to-end driving applications. The dataset DDD17 accompanying this paper is the first open dataset of annotated DAVIS driving recordings. DDD17 has over 12 h of a 346x260 pixel DAVIS sensor recording highway and city driving in daytime, evening, night, dry and wet weather conditions, along with vehicle speed, GPS position, driver steering, throttle, and brake captured from the car's on-board diagnostics interface. As an example application, we performed a preliminary end-to-end learning study of using a convolutional neural network that is trained to predict the instantaneous steering angle from DVS and APS visual data.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Cityscapes [270] X X Vehicle Berkley DeepDrive [271] X X Vehicle X X Mapillary [272] X X Vehicle X X Oxford RobotCar [49] X X Vehicle X X X KITTI [174] X X X X Vehicle H3D [273] X X X Vehicle ApolloScape [274] X X X X Vehicle nuScenes [175] X X X X Vehicle X X Udacity [275] X X X X Vehicle DDD17 [85”
Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, background): “One fruitful direction is the collection of dynamic vision sensor (DVS) benchmarks (Orchard et al., 2015a; Serrano-Gotarredona and Linares-Barranco, 2015; Hu et al., 2016; Liu et al., 2016), in particular for relevant use cases such as automated driving (Binas et al., 2017; Sironi et al., 2018).”
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars (2018, methodology): “To predict steering angles from event images we use the publicly available DAVIS Driving Dataset 2017 (DDD17) [22].”
DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, unknown): “DDD17 [35] and DDD20 [36] are datasets comprising many hours of driving data from a monochrome DAVIS346 camera featuring various vehicle control data.”
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras (2018, background): “In addition, Zhu et al. [30] provide ﬂying, driving and handheld sequences with ground truth camera pose and depth maps, and Binas et al. [7] provide long driving sequences with ground truth measurements from the vehicle such as steering angle and GPS position.”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “0 HDD [144] 3 X X X X X X X X X 104h Academic Brain4Cars [145] 1 X X X X 1180 miles Academic Li-Vi [95], [146] 1 X X X X X 10h Academic DDD17 [147] 1* X X X X X X 12h CC-BY-NC-SA-4.”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, background): “4) DDD17: As the first end-to-end dynamic and activepixel vision sensors (DAVISs) driving dataset, DDD17 [59] has more than 12 h of DAVIS sensor data under different scenarios and different weather conditions, as well as vehicle control information, such as steering, throttle, and brake.”
ESIM: an Open Event Camera Simulator (2018, background): “While N-CARS [18] speciﬁcally focuses on car recognition, DDD17 [21] provides the vehicle speed, GPS position, driver steering, throttle, and brake captured from the car’s on-board diagnostics interface.”
Event-Based Vision: A Survey (2019, background): “Sample applications include classiﬁcation [134], [135], image reconstruction [8], [102], steering angle prediction [91], [136], and estimation of optical ﬂow [22], [103], [137], depth [137] or ego-motion [103].”
Learning to Detect Objects with a 1 Megapixel Event Camera (2020, background): “Despite the increasing effort of the community [29, 30, 31, 32], very few datasets provide ground-truth for object detection.”
The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception (2018, background): “[5] provide a large dataset of a DAVIS 346B mounted behind the windshield of a car, with 12 hours of driving, intended for end to end learning of various driving related tasks.”