PatchBatch: A Batch Augmented Loss for Optical Flow

By David Gadot, Lior Wolf. 2015

105 citations recorded by the source corpus.

We propose a new pipeline for optical flow computation, based on Deep Learning techniques. We suggest using a Siamese CNN to independently, and in parallel, compute the descriptors of both images. The learned descriptors are then compared efficiently using the L2 norm and do not require network processing of patch pairs. The success of the method is based on an innovative loss function that computes higher moments of the loss distributions for each training batch. Combined with an Approximate Nearest Neighbor patch matching method and a flow interpolation technique, state of the art performance is obtained on the most challenging and competitive optical flow benchmarks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016, background): “Gadot & Wolf [12] and Bailer et al.”
Video Frame Interpolation via Adaptive Separable Convolution (2017, methodology): “Our research is inspired by the success of applying deep learning to optical ﬂow estimation [2, 12, 16, 19, 50, 51, 52], artistic style transfer [17, 22, 28], and image enhancement [6, 9, 10, 46, 47, 55, 57, 62, 65].”
Video Frame Interpolation via Adaptive Convolution (2017, methodology): “Our work is inspired by the success of deep learning in solving not only difficult visual understanding problems [16, 20, 26, 28, 39, 40, 42, 45, 54, 60, 64] but also other computer vision problems like optical flow estimation [9, 14, 19, 48, 49, 52], style transfer [11, 15, 23, 30, 50], and image e”
SDC – Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks (2019, methodology): “The 200 training images from KITTI [11], are randomly split into a subset for actual training (70 %), one for validation (20 %), and one for testing in our experiments section (10 %).”
UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss (2017, methodology): “A number of other deep architectures based on supervised learning have been proposed, leveraging various paradigms, including patch matching (Gadot and Wolf 2016), discrete optimization (G¨uney and Geiger 2016), and coarse-to-ﬁne estimation (Ranjan and Black 2017).”
Context-Aware Synthesis for Video Frame Interpolation (2018, result): “Similar to many other computer vision problems, deep learning approaches are used in various high-quality optical flow algorithms [9, 11, 43, 47, 49].”
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (2019, methodology): “Because occluded pixels do not have correspondences in the other frame, several approaches [5, 12, 16,23,32] aim to filter out these outliers to minimize their ill effects and ap ply post-processing to refine the estimates [22, 33, 48, 67].”
Deep Discrete Flow (2016, methodology): “In [49], Siamese networks for optical flow computation have been combined with winner-takes-all matching and smoothing of the resulting correspondence field.”
Self-Supervised Visual Descriptor Learning for Dense Correspondence (2017, background): “[4], and Gadot and Wolf [7]) to identify dense correspondences in images”
Fast Optical Flow Using Dense Inverse Search (2016, methodology): “[38] learn descriptors appropriate for optical flow using siamese CNNs.”
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume (2017, unknown): “PatchBatch [17] - 5.”