The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM

By Elias Mueggler, Henri Rebecq, Guillermo Gallego, Tobi Delbruck, Davide Scaramuzza. Int. J. Robotics Res., 2016

735 citations recorded by the source corpus.

New vision sensors, such as the dynamic and active-pixel vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array. These sensors have great potential for high-speed robotics and computer vision because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and very high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called “events”) and synchronous grayscale frames. For this purpose, we present and release a collection of datasets captured with a DAVIS in a variety of synthetic and real environments, which we hope will motivate research on new algorithms for high-speed and high-dynamic-range robotics and computer-vision applications. In addition to global-shutter intensity images and asynchronous events, we provide inertial measurements and ground-truth camera poses from a motion-capture system. The latter allows comparing the pose accuracy of ego-motion estimation algorithms quantitatively. All the data are released both as standard text files and binary files (i.e. rosbag). This paper provides an overview of the available data and describes a simulator that we release open-source to create synthetic event-camera data.

Subfields: Aerospace Engineering, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/8da267f7dec1cedc59f9c9aa7c4e1f88e3dcc62b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Electrical and Electronic Engineering

Citations in the corpus

Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High-Speed Scenarios (2017, methodology): “Comparison of the proposed approach, using frames (Fr), events (E), and IMU (I), on three datasets from the Event Camera Dataset [14].”
The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception (2018, background): “While the more recent works evaluate based on public datasets such as [4], the majority are evaluated on small datasets generated solely for the paper, making comparisons of performance difficult.”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “Tonic: Event-Based Datasets and Transformations.”
v2e: From Video Frames to Realistic DVS Events (2021, methodology): “Thus, DVS simulators[15, 21, 26] and transfer learning methods such as [26, 8] were developed to exploit existing intensity and mixed modality [11] frame datasets.”
Complex urban dataset with multi-level sensors from highly diverse urban environments (2019, background): “Event camera datasets involve recently introduced event cameras (Mueggler et al., 2017; Zhu et al., 2018) for the purpose of overcoming illumination and motion blur issues caused by conventional optical images.”
Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, background): “Instead, they should be speciﬁcally designed to show the advantages of the neuromorphic approach (Orchard et al., 2015a; Tan et al., 2015; Barranco et al., 2016; Hu et al., 2016; Liu et al., 2016; Binas et al., 2017; Mueggler et al., 2017).”
ESIM: an Open Event Camera Simulator (2018, methodology): “[10] was introduced as a benchmark for event-based visual (inertial) odometry.”
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras (2018, neutral): “[15], where we received similar results.”
Predicting the Future from First Person (Egocentric) Vision: A Survey (2021, methodology): “Another important type of devices to mention is event cameras, such as the dynamic and active-pixel vision sensor (DAVIS) [45] which can be used for computer vision tasks such as SLAM or pose estimation [46].”
High Speed and High Dynamic Range Video with an Event Camera (2019, methodology): “As is standard [58], [59], [60], we use sequences from the Event Camera Dataset [38], which contain events, frames, and IMU measurements from a DAVIS240C [25] sensor.”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, methodology): “This figure shows an actual output sample from our method, operating on a real event sequence from a publicly available dataset [30].”
HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification (2018, background): “The authors of [44] released an event-based dataset adapted to test visual odometry algorithms.”
Event-Based Vision: A Survey (2019, background): “Several event representations (Section 3.1) of the slider_depth sequence [81].”
Bringing a Blurry Frame Alive at High Frame-Rate With an Event Camera (2018, methodology): “3, we give an example for manually chosen and automatically optimized results on dataset from [20].”