The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception

By Alex Zihao Zhu, Dinesh Thakur, Tolga Özaslan, Bernd Pfrommer, Vijay R. Kumar, Kostas Daniilidis. IEEE Robotics and Automation Letters, 2018

621 citations recorded by the source corpus.

Event-based cameras are a new passive sensing modality with a number of benefits over traditional cameras, including extremely low latency, asynchronous data acquisition, high dynamic range, and very low power consumption. There has been a lot of recent interest and development in applying algorithms to use the events to perform a variety of three-dimensional perception tasks, such as feature tracking, visual odometry, and stereo depth estimation. However, there currently lacks the wealth of labeled data that exists for traditional cameras to be used for both testing and development. In this letter, we present a large dataset with a synchronized stereo pair event based camera system, carried on a handheld rig, flown by a hexacopter, driven on top of a car, and mounted on a motorcycle, in a variety of different illumination levels and environments. From each camera, we provide the event stream, grayscale images, and inertial measurement unit (IMU) readings. In addition, we utilize a combination of IMU, a rigidly mounted lidar system, indoor and outdoor motion capture, and GPS to provide accurate pose and depth images for each camera at up to 100 Hz. For comparison, we also provide synchronized grayscale images and IMU readings from a frame-based stereo camera system.

Subfields: Aerospace Engineering, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/f1e92f09209c7f50e05599c7551520ca129a6de4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Electrical and Electronic Engineering

Citations in the corpus

Unsupervised Event-Based Learning of Optical Flow, Depth, and Egomotion (2018, methodology): “We tested our optical ﬂow network on the indoor ﬂying and outdoor day sequences from MVSEC, with the ground truth provided by [26].”
v2e: From Video Frames to Realistic DVS Events (2021, methodology): “The second task is a car detection task us-ing the MVSEC dataset [35].”
EKLT: Asynchronous Photometric Feature Tracking Using Events and Frames (2018, methodology): “2018),3 “outdoor_day1” from the Multi-Vehicle Stereo Event Camera Dataset (MVSEC) (Zhu et al. 2018) and “outdoor_forward5” from the UZH-FPV DroneRacingDataset (Delmerico et al.”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, background): “A high-dynamic-range reconstruction from an event camera mounted on a car driving at night [54].”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “MVSEC [101] Spikes, frames and optical flow from stereo cameras for indoor and outdoor scenarios.”
DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, methodology): “To the best of our knowledge, MVSEC [2] is the only dataset with a stereo event camera in driving scenarios.”
End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, background): “, CNNs [36, 59, 68, 69].”
High Speed and High Dynamic Range Video with an Event Camera (2019, background): “There are many publicly available event camera datasets featuring HDR scenes [5], [38], [43].”
Complex urban dataset with multi-level sensors from highly diverse urban environments (2019, background): “Event camera datasets involve recently introduced event cameras (Mueggler et al., 2017; Zhu et al., 2018) for the purpose of overcoming illumination and motion blur issues caused by conventional optical images.”
ESIM: an Open Event Camera Simulator (2018, methodology): “A number of event datasets has been speciﬁcally designed for low-level vision tasks such as visual (inertial) odometry [9, 10, 11], optic ﬂow estimation [12, 13] or depth from stereo [14, 11].”
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras (2018, neutral): “[30] provide flying, driving and handheld sequences with ground truth camera pose and depth maps, and Binas et al.”
Event-Based Vision: A Survey (2019, methodology): “r not. Let us present the methods using these criteria. Normal vs. Full Flow: Early works, such as [29], [30], [88], estimated normal optical ﬂow: the component of the 10 (a) Scene Indoor Flying from [182] (b) Sparse optical ﬂow Figure 5. Example of optical ﬂow estimation with an event camera. (a) E”
Learning to Detect Objects with a 1 Megapixel Event Camera (2020, methodology): “Despite the increasing effort of the community [29, 30, 31, 32], very few datasets provide ground-truth for object detection.”