HFirst: A Temporal Approach to Object Recognition

By Garrick Orchard, Cedric Meyer, Ralph Etienne-Cummings, Christoph Posch, Nitish V. Thakor, Ryad Benosman. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015

306 citations recorded by the source corpus.

This paper introduces a spiking hierarchical model for object recognition which utilizes the precise timing information inherently present in the output of biologically inspired asynchronous address event representation (AER) vision sensors. The asynchronous nature of these systems frees computation and communication from the rigid predetermined timing enforced by system clocks in conventional systems. Freedom from rigid timing constraints opens the possibility of using true timing to our advantage in computation. We show not only how timing can be used in object recognition, but also how it can in fact simplify computation. Specifically, we rely on a simple temporal-winner-take-all rather than more computationally intensive synchronous operations typically used in biologically inspired neural networks for object recognition. This approach to visual computation represents a major paradigm shift from conventional clocked systems and can find application in other sensory modalities and computational tasks. We showcase effectiveness of the approach by achieving the highest reported accuracy to date (97.5% ± 3.5%) for a previously published four class card pip recognition task and an accuracy of 84.9% ± 1.9% for a new more difficult 36 class character recognition task.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/9b2c1eccbc3ba4790a2a3cb0f4188f565d4b5673

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification (2018, methodology): “The baselines methods we consider are HOTS [30], H-First [50] and Spiking Neural Networks ( SNN ) [33, 46].”
Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades (2015, methodology): “…early Neuromorphic datasets for recognition include the four class card pip dataset from Pérez-Carrasco et al. (2013), the 36 character dataset from Orchard et al. (2015), the four class silhouette orientation dataset from Pérez-Carrasco et al. (2013), and the three class posture dataset from Zhao…”
HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition (2017, methodology): “This task, which will be referred as the letters & digits recognition task (Section 4.2), has already been tack-led by [14].”
A review of learning in biologically plausible spiking neural networks (2019, unknown): “Orchard et al. (Orchard et al., 2015) have assigned an applied input to the class related to the output neuron that fires first, and they have reported this output encoding has achieved a good performance.”
Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification (2017, methodology): “This would allow the deployment of deep spiking networks in combination with an event-based sensor for real-world applications (Orchard et al., 2015a; Serrano-Gotarredona et al., 2015; Kiselev et al., 2016).”
End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, methodology): “Despite their differences, both paradigms have been successfully applied on various vision tasks, including tracking [19,21,40,42], depth estimation [3,52,67], visual odometry [27, 54, 57, 66], recognition [29, 44], and optical ﬂow estimation [7, 69].”
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars (2018, background): “For example [16, 17, 18] use neural networks on event data to recognize cards of a deck (4 classes), faces (7 classes) or characters (36 classes).”
Skimming Digits: Neuromorphic Classification of Spike-Encoded Images (2016, methodology): “This paper proposes and investigates four approaches to determining the output in a multi-class problem using SKIM, primarily applied to the N-MNIST dataset and also applicable to the multi-class classiﬁcation problems in the N-Caltech101 dataset introduced in Orchard et al. (2015a).”
High Speed and High Dynamic Range Video with an Event Camera (2019, methodology): “patterns from a stream of events with minimal latency (HFIRST [48]), conventional machine learning techniques combined with novel event representations such as time”
Event-Based Vision: A Survey (2019, methodology): “In the simplest case, each template can be obtained from an individual sample, but such templates are sensitive to noise in the sample data [15].”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, background): “While one line of work focuses on spiking neural architectures (SNNs) to recognize patterns from a stream of events with minimal latency (H-FIRST [33]), conventional machine learning techniques combined with novel event representations such as time surfaces (HOTS [21]) have shown the most promising ”