Events-To-Video: Bringing Modern Computer Vision to Event Cameras
By Henri Rebecq, René Ranftl, Vladlen Koltun, Davide Scaramuzza. Computer Vision and Pattern Recognition, 2019
484 citations recorded by the source corpus.
Event cameras are novel sensors that report brightness changes in the form of asynchronous "events" instead of intensity frames. They have significant advantages over conventional cameras: high temporal resolution, high dynamic range, and no motion blur. Since the output of event cameras is fundamentally different from conventional cameras, it is commonly accepted that they require the development of specialized algorithms to accommodate the particular nature of events. In this work, we take a different view and propose to apply existing, mature computer vision techniques to videos reconstructed from event data. We propose a novel, recurrent neural network to reconstruct videos from a stream of events and train it on a large amount of simulated event data. Our experiments show that our approach surpasses state-of-the-art reconstruction methods by a large margin (> 20%) in terms of image quality. We further apply off-the-shelf computer vision algorithms to videos reconstructed from event data on tasks such as object classification and visual-inertial odometry, and show that this strategy consistently outperforms algorithms that were specifically designed for event data. We believe that our approach opens the door to bringing the outstanding properties of event cameras to an entirely new range of tasks.
Subfields: Electrical and Electronic Engineering, Materials Chemistry
View Paper: https://www.semanticscholar.org/paper/a23fbf6e7c224e696662b948163cb27e52188356
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Electrical and Electronic Engineering
- Davide Scaramuzza
- Henri Rebecq
- Shih-Chii Liu
- Tobi Delbruck
- Vladlen Koltun
- René Ranftl
- Yuhuang Hu
- Daniel Gehrig
In Materials Chemistry
Citations in the corpus
- High Speed and High Dynamic Range Video with an Event Camera (2019, methodology): “We now compare our network (based on stacked ConvLSTMs [27]) to the preliminary version of this work [30] based on a vanilla RNN, focusing on the generalization ability of both networks to the duration of the event windows used (or, equivalently, the number of events in each window).”
- v2e: From Video Frames to Realistic DVS Events (2021, methodology): “Our experiments use the event voxel grid method to convert N events into a 3D representation with size H × W × D [27, 11] to use as the network input for the Sec.”