Unsupervised convolutional neural networks for motion estimation

By Aria Ahmadi, Ioannis Patras. 2016

108 citations recorded by the source corpus.

Traditional methods for motion estimation estimate the motion field F between a pair of images as the one that minimizes a predesigned cost function. In this paper, we propose a direct method and train a Convolutional Neural Network (CNN) that when, at test time, is given a pair of images as input it produces a dense motion field F at its output layer. In the absence of large datasets with ground truth motion that would allow classical supervised training, we propose to train the network in an unsupervised manner. The proposed cost function that is optimized during training, is based on the classical optical flow constraint. The latter is differentiable with respect to the motion field and, therefore, allows backpropagation of the error to previous layers of the network. Our method is tested on both synthetic and real image sequences and performs similarly to the state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016, neutral): “Following FlowNet, several papers have studied optical flow estimation with CNNs: featuring a 3D CNN [30], unsupervised learning [1, 33], carefully designed rotationally invariant architectures [28], or a pyramidal approach based on the coarse-to-fine idea of variational methods [20].”
Optical Flow Estimation Using a Spatial Pyramid Network (2016, background): “There have also been several attempts at estimating optical flow using unsupervised learning [3, 47].”
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (2019, background): “Fl ow N et [1 4] 2.”
Occlusion Aware Unsupervised Learning of Optical Flow (2017, background): “Later, [30, 41, 1] adopt a similar unsupervised learning architecture with a more detailed performance study on multiple optical flow benchmark datasets.”
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation (2016, methodology): “…can be expressed as I ′ t +1 ( x, y ) = I{ I t +1 ( x + ∆ t +1 x, y + ∆ t +1 y ) , or more concisely We adopt a multi-scale design to represent the ﬂow, which has been shown to be effective in classical methods [10, 2] and also in more recently proposed spatial trans-former techniques [11, 1, 9].”
DenseNet for dense flow (2017, methodology): “Though [1, 2, 5, 6] are totally different approaches, they all use variants of one architecture, the “FlowNet Simple” network [1].”
UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss (2017, methodology): “This limitation is shared by previous works employing an unsupervised proxy loss (Ahmadi and Patras 2016; Yu, Harley, and Der-panis 2016; Ren et al. 2017).”