EKLT: Asynchronous Photometric Feature Tracking Using Events and Frames

By Daniel Gehrig, Henri Rebecq, Guillermo Gallego, Davide Scaramuzza. International Journal of Computer Vision, 2018

330 citations recorded by the source corpus.

We present EKLT, a feature tracking method that leverages the complementarity of event cameras and standard cameras to track visual features with high temporal resolution. Event cameras are novel sensors that output pixel-level brightness changes, called “events”. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the same scene pattern can produce different events depending on the motion direction, establishing event correspondences across time is challenging. By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction. Our method extracts features on frames and subsequently tracks them asynchronously using events, thereby exploiting the best of both types of data: the frames provide a photometric representation that does not depend on motion direction and the events provide updates with high temporal resolution. In contrast to previous works, which are based on heuristics, this is the first principled method that uses intensity measurements directly, based on a generative event model within a maximum-likelihood framework. As a result, our method produces feature tracks that are more accurate than the state of the art, across a wide variety of scenes.

Subfields: Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/7aeb6b4a558da31c212794425b8f86e6846308f0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Electrical and Electronic Engineering

Citations in the corpus

Bringing a Blurry Frame Alive at High Frame-Rate With an Event Camera (2018, background): “Although several works try to explore the advantages of the high temporal resolution provided by event cameras [41, 13, 26, 43, 42, 8, 15, 30], how to make the best use of the event camera has not yet been fully investigated.”
Low-latency automotive vision with event cameras (2024, unknown): “For this reason, several methods have combined events and frames for moving-object detections 79 , tracking 80 , computational photography 22,81,82 and monocular depth estimation 40 .”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, background): “Such specialized algorithms have demonstrated impressive performance in applications ranging from low-level vision tasks, such as visual odometry [18, 39, 53, 38, 41], feature tracking [20, 52, 15] and optical flow [5, 3, 47, 55, 46], to high-level tasks such as object classification [35, 21, 45] an”
High Speed and High Dynamic Range Video with an Event Camera (2019, background): “They showed how to reconstruct a single image from a large set of events collected by an event camera moving through a static scene and exploited the fact that every event provides one equation relating the intensity gradient and optic flow through brightness constancy [18].”
Event-Based Vision: A Survey (2019, methodology): “The above interpretation may be taken into account to design physically-grounded event-based algorithms, such as [7], [23], [24], [28], [62], [63], [64], [65], as opposed to algorithms that simply process events as a collection of points with vague photometric meaning.”
End-to-End Learning of Representations for Asynchronous Event-Based Data (2019, methodology): “Despite their differences, both paradigms have been successfully applied on various vision tasks, including tracking [19,21,40,42], depth estimation [3,52,67], visual odometry [27, 54, 57, 66], recognition [29, 44], and optical ﬂow estimation [7, 69].”
DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, unknown): “The fusion of frames and events has proven to be beneficial in several prior works [13]–[17].”