End-to-End Learning of Representations for Asynchronous Event-Based Data

By Daniel Gehrig, Antonio Loquercio, Konstantinos G. Derpanis, Davide Scaramuzza. IEEE International Conference on Computer Vision, 2019

454 citations recorded by the source corpus.

Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events”. They have appealing advantages over frame based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatio-temporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations by means of strictly differentiable operations. Our framework comes with two main advantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/778e0ce80e523cf99ea9c746bcc1a1ffaa87be4c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Learning to Detect Objects with a 1 Megapixel Event Camera (2020, unknown): “with [50, 51, 52].”
v2e: From Video Frames to Realistic DVS Events (2021, background): “Compared to the second group of the table, our ﬁne-tuned model accuracy is also higher than recent literature [31, 1, 9].”
Event-Based Vision: A Survey (2019, methodology): “A general framework for converting event data into some of the above grid-based representations is presented in [83].”