Fast Optical Flow Using Dense Inverse Search

By Till Kroeger, Radu Timofte, Dengxin Dai, Luc Van Gool. 2016

384 citations recorded by the source corpus.

Most recent works in optical flow extraction focus on the accuracy and neglect the time complexity. However, in real-life visual applications, such as tracking, activity detection and recognition, the time complexity is critical. We propose a solution with very low time complexity and competitive accuracy for the computation of dense optical flow. It consists of three parts: (1) inverse search for patch correspondences; (2) dense displacement field creation through patch aggregation along multiple scales; (3) variational refinement. At the core of our Dense Inverse Search-based method (DIS) is the efficient search of correspondences inspired by the inverse compositional image alignment proposed by Baker and Matthews (2001, 2004). DIS is competitive on standard optical flow benchmarks. DIS runs at 300 Hz up to 600 Hz on a single CPU core (1024 \(\times \) 436 resolution. 42 Hz/46 Hz when including preprocessing: disk access, image re-scaling, gradient computation. More details in Sect. 3.1.), reaching the temporal resolution of human’s biological vision system. It is order(s) of magnitude faster than state-of-the-art methods in the same range of accuracy, making DIS ideal for real-time applications.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016, neutral): “70 – 140 – DIS-Fast [15] 5.”
Optical Flow Estimation Using a Spatial Pyramid Network (2016, methodology): “Most recently DIS-Fast [26] is a GPU method that is signiﬁcantly faster than previous methods but is also signiﬁcantly less accurate.”
Online Real-Time Multiple Spatiotemporal Action Localisation and Prediction (2016, methodology): “In all settings we generate results by running our framework in five different ‘modes’: 1) Appearance (A) – only RGB video frames are processed by a single SSD; 2) Realtime flow (RTF) – optical flow images are computed in realtime [16] and fed to a single SSD; 3) A+RTF: both RGB and real-time optica”
Action Tubelet Detector for Spatio-Temporal Action Localization (2017, methodology): “[30] perform action localization in real-time using (a) the efficient SSD detector, (b) a fast method [14] to estimate the optical flow for the motion stream, and (c) an online linking algorithm.”
Semantic Video Segmentation by Gated Recurrent Flow Propagation (2016, methodology): “For validation and testing we used DIS [23] which gave a higher accuracy.”
Context-Aware Synthesis for Video Frame Interpolation (2018, methodology): “To determine the former, we estimated the optical flow between the first and the last patch in each triplet using DIS [21].”
End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners (2018, background): “Typical examples include the detection of roads [62], traffic lights [63], cars and pedestrians [64,65,1,2], and the tracking of such objects [66,67,68].”
Video Acceleration Magnification (2017, background): “The Lagrangian approach uses optical flow which is expensive and an unsolved research topic in its own [6, 14, 21].”
Jump (2016, background): “The requirements of speed and visual quality are often at odds with each other, as most fast techniques oversmooth at flow discontinuities [Kroeger et al. 2016], and most edge-aware techniques [Krähenbühl and Koltun 2012; Revaud et al.”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “The approach proposed by Kroeger et al. (2016) allows to trade-off accuracy and computational time.”