From Motion Blur to Motion Flow: A Deep Learning Solution for Removing Heterogeneous Motion Blur

By Dong Gong, Jie Yang, Lingqiao Liu, Yanning Zhang, Ian David Reid, Chunhua Shen, Anton van den Hengel, Javen Qinfeng Shi. Computer Vision and Pattern Recognition, 2016

436 citations recorded by the source corpus.

Removing pixel-wise heterogeneous motion blur is challenging due to the ill-posed nature of the problem. The predominant solution is to estimate the blur kernel by adding a prior, but extensive literature on the subject indicates the difficulty in identifying a prior which is suitably informative, and general. Rather than imposing a prior based on theory, we propose instead to learn one from the data. Learning a prior over the latent image would require modeling all possible image content. The critical observation underpinning our approach, however, is that learning the motion flow instead allows the model to focus on the cause of the blur, irrespective of the image content. This is a much easier learning task, but it also avoids the iterative process through which latent image priors are typically applied. Our approach directly estimates the motion flow from the blurred image through a fully-convolutional deep neural network (FCN) and recovers the unblurred image from the estimated motion flow. Our FCN is the first universal end-to-end mapping from the blurred image to the dense motion flow. To train the FCN, we simulate motion flows to generate synthetic blurred-image-motion-flow pairs thus avoiding the need for human labeling. Extensive experiments on challenging realistic blurred images demonstrate that the proposed method outperforms the state-of-the-art.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3cf69d0d964eef168df8eeff4431cebfc23fafe3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better (2019, methodology): “[8] used a fully convolutional network to estimate the motion flow.”
Bringing a Blurry Frame Alive at High Frame-Rate With an Event Camera (2018, background): “7, we show the quantitative and qualitative comparisons with the state-of-the-art image deblurring approaches [33, 22, 9, 11, 35, 40, 21], and the video reconstruction method [29] on our synthetic dataset, respectively.”
DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks (2017, methodology): “[36] use CNN to estimate blur kernel, Chakrabarti [6] predicts complex Fourier coefficients of motion kernel to perform non-blind deblurring in Fourier space whereas Gong [9] use fully convolutional network to move for motion flow estimation.”
Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring (2020, background): “Method [24] [23] [5] [3] [13] [8] [20] [27] [2] [26] [9] Ours(a) Ours(b)”
Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration (2019, methodology): “Recently, CNN-based methods [9, 20, 28, 38, 42] achieve good performance for this task.”
Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections (2019, methodology): “[5] utilized a fully convolutional network to estimate the dense heterogeneous motion flow from the blurred image and still used the method of [39] to recover the latent image.”
Deep Image Deblurring: A Survey (2022, methodology): “…2020; Schuler et al. 2015; Sun et al. 2015) train 2D CNNs to directly recover sharp images without kernel estimation steps (Aljadaany et al. 2019; Gong et al. 2017; Lu et al. 2019; Madam Nimisha et al. 2018; Mus-taniemi et al. 2019; Nimisha et al. 2017; Shen et al. 2019; Zhang et al. 2019,?).”
Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network (2020, background): “Inspired by the successes of machine learning in many computer vision tasks [9, 10, 39, 40], some learning based approaches are also proposed.”
Attention-Guided Network for Ghost-Free High Dynamic Range Imaging (2019, background): “Inspired by the successes of the deep neural networks (DNNs) in many image restoration tasks [18, 7, 40, 8], some deep learning-based approaches [15, 37, 38] have been proposed recently to improve the HDR image composition process.”