Learning to Detect Objects with a 1 Megapixel Event Camera

By Etienne Perot, Pierre de Tournemire, Davide Oscar Nitti, Jonathan Masci, Amos Sironi. 2020

351 citations recorded by the source corpus.

Event cameras encode visual information with high temporal precision, low data-rate, and high-dynamic range. Thanks to these characteristics, event cameras are particularly suited for scenarios with high motion, challenging lighting conditions and requiring low latency. However, due to the novelty of the field, the performance of event-based systems on many vision tasks is still lower compared to conventional frame-based solutions. The main reasons for this performance gap are: the lower spatial resolution of event sensors, compared to frame cameras; the lack of large-scale training datasets; the absence of well established deep learning architectures for event-based processing. In this paper, we address all these problems in the context of an event-based object detection task. First, we publicly release the first high-resolution large-scale dataset for object detection. The dataset contains more than 14 hours recordings of a 1 megapixel event camera, in automotive scenarios, together with 25M bounding boxes of cars, pedestrians, and two-wheelers, labeled at high frequency. Second, we introduce a novel recurrent architecture for event-based detection and a temporal consistency loss for better-behaved training. The ability to compactly represent the sequence of events into the internal memory of the model is essential to achieve high accuracy. Our model outperforms by a large margin feed-forward event-based architectures. Moreover, our method does not require any reconstruction of intensity images from events, showing that training directly from raw events is possible, more efficient, and more accurate than passing through an intermediate intensity image. Experiments on the dataset introduced in this work, for which events and gray level images are available, show performance on par with that of highly tuned and studied frame-based detectors.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets (2022, background): “Current large-scale, high-resolution datasets are of single object classes or contain many similar images [Fregin et al. 2018; Perot et al. 2020; Zhang et al. 2020].”
v2e: From Video Frames to Realistic DVS Events (2021, background): “A recent paper [24] only observed this phenomenon qualitatively.”
Neuromorphic computing at scale (2025, neutral): “Several datasets have been developed specifically for benchmarking spiking systems 87–93 ; however, these are still not widely adopted 30 .”
DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, background): “was recently extended with a high resolution (1 MP) event camera [33] for the same task.”
Low-latency automotive vision with event cameras (2024, neutral): “64) and the YOLO series 34,65,66 , and the widespread adoption of these methods in automotive settings, event-based object detection research has focused on leveraging the available models on dense, image-like event representations 19,26–29 , 38 .”