HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification

By Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, Ryad Benosman. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

552 citations recorded by the source corpus.

Event-based cameras have recently drawn the attention of the Computer Vision community thanks to their advantages in terms of high temporal resolution, low power consumption and high dynamic range, compared to traditional frame-based cameras. These properties make event-based cameras an ideal choice for autonomous vehicles, robot navigation or UAV vision, among others. However, the accuracy of event-based object classification algorithms, which is of crucial importance for any reliable system working in real-world conditions, is still far behind their frame-based counterparts. Two main reasons for this performance gap are: 1. The lack of effective low-level representations and architectures for event-based object classification and 2. The absence of large real-world event-based datasets. In this paper we address both problems. First, we introduce a novel event-based feature representation together with a new machine learning architecture. Compared to previous approaches, we use local memory units to efficiently leverage past temporal information and build a robust event-based representation. Second, we release the first large real-world event-based dataset for object classification. We compare our method to the state-of-the-art with extensive experiments, showing better classification performance and real-time computation.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/a60f20adc4fc9f363465de327dc2f3f1ab7b624c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, background): “N-CARS [34] contains very short (100 ms) sequences for binary classiﬁcation of cars.”
Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, methodology): “The HATS method (Sironi et al., 2018) divides the image into a regular grid of cells, and smoothes all time surfaces within each cell and specified time window.”
Low-latency automotive vision with event cameras (2024, unknown): “However, these architectures lack efficient learning rules and thus do not yet scale to complex tasks and datasets 42,70–74 .”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, methodology): “For the baselines, we report directly the accuracy provided in [45].”
Learning to Detect Objects with a 1 Megapixel Event Camera (2020, methodology): “Some of these methods, such as Spiking Neural Networks [14, 15, 16, 17], exploit the sparsity of the data and can be applied event by event, to preserve the temporal resolution of the events stream [15, 18, 19, 20].”
v2e: From Video Frames to Realistic DVS Events (2021, background): “Compared to the second group of the table, our ﬁne-tuned model accuracy is also higher than recent literature [31, 1, 9].”
End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, methodology): “In the case of HATS [59], the exponential time surfaces can be interpreted as a local convolution of the spike train with an exponential kernel.”
High Speed and High Dynamic Range Video with an Event Camera (2019, methodology): “For example, [13], [14] tackle the task of object classification from event data as follows.”
ESIM: an Open Event Camera Simulator (2018, background): “While N-CARS [18] specifically focuses on car recognition, DDD17 [21] provides the vehicle speed, GPS position, driver steering, throttle, and brake captured from the car’s on-board diagnostics interface.”