Unsupervised Event-Based Learning of Optical Flow, Depth, and Egomotion
By Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, Kostas Daniilidis. Computer Vision and Pattern Recognition, 2018
710 citations recorded by the source corpus.
In this work, we propose a novel framework for unsupervised learning for event cameras that learns motion information from only the event stream. In particular, we propose an input representation of the events in the form of a discretized volume that maintains the temporal distribution of the events, which we pass through a neural network to predict the motion of the events. This motion is used to attempt to remove any motion blur in the event image. We then propose a loss function applied to the motion compensated event image that measures the motion blur in this image. We train two networks with this framework, one to predict optical flow, and one to predict egomotion and depths, and evaluate these networks on the Multi Vehicle Stereo Event Camera dataset, along with qualitative results from a variety of different scenes.
Subfields: Cellular and Molecular Neuroscience, Electrical and Electronic Engineering
View Paper: https://www.semanticscholar.org/paper/e2d75694a7a1b6ee6d6f30885bfd1d8466ba105e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cellular and Molecular Neuroscience
- Davide Scaramuzza
- Garrick Orchard
- Daniel Gehrig
- Amos Sironi
- Chiara Bartolozzi
- Guillermo Gallego
- Andrew J. Davison
- Stefan Leutenegger
In Electrical and Electronic Engineering
- Davide Scaramuzza
- Daniel Gehrig
- Guillermo Gallego
- Kostas Daniilidis
- Tobi Delbruck
- Henri Rebecq
- Antonio Loquercio
- Garrick Orchard
Citations in the corpus
- End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, methodology): “[70] converted events into a spatiotemporal voxel-grid.”
- Event-Based Vision: A Survey (2019, methodology): “[22], [114] Full Dense ANN No Ye et al.”
- Learning to Detect Objects with a 1 Megapixel Event Camera (2020, methodology): “Here we compare the commonly used Histograms of events [48, 22], Time Surfaces [49], and Event Volumes [23].”
- DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, methodology): “our baseline implementation uses the voxel grid representation [49] that contains events within a duration of 50 milliseconds.”