Flow-edge Guided Video Completion

By Chen Gao, Ayush Saraf, Jia-Bin Huang, Johannes Peter Kopf. 2020

207 citations recorded by the source corpus.

We present a new flow-based video completion algorithm. Previous flow completion methods are often unable to retain the sharpness of motion boundaries. Our method first extracts and completes motion edges, and then uses them to guide piecewise-smooth flow completion with sharp edges. Existing methods propagate colors among local flow connections between adjacent frames. However, not all missing regions in a video can be reached in this way because the motion boundaries form impenetrable barriers. Our method alleviates this problem by introducing non-local flow connections to temporally distant frames, enabling propagating video content over motion boundaries. We validate our approach on the DAVIS dataset. Both visual and quantitative results show that our method compares favorably against the state-of-the-art algorithms.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, methodology): “With our robust per-frame segmentation of hand masks, we use the recent flow-guided video inpainting algorithm [12] to completely remove the hands such that we could see the original content occluded by hands in the videos.”
FlowFormer: A Transformer Architecture for Optical Flow (2022, background): “In many downstream video tasks, such as action recognition [46,37,61], video inpainting [28,51,13], video super-resolution [30,5,39], and frame interpolation [52,33,20], optical flow serves as a fundamental component providing dense correspondences as valuable clues for prediction.”
Space-time Neural Irradiance Fields for Free-Viewpoint Video (2020, methodology): “[38]; (“Inpainted”) its inpainted version, where disoccluded empty pixels in the 2D rendered images are inpainted using a recent video inpainting method [19]; and (“NeRF-T”) a version of NeRF with an extra time parameter, which is our model trained with only the color reconstruction loss (4).”