Learning a convolutional neural network for non-uniform motion blur removal

By Jian Sun, Wenfei Cao, Zongben Xu, Jean Ponce. Computer Vision and Pattern Recognition, 2015

920 citations recorded by the source corpus.

In this paper, we address the problem of estimating and removing non-uniform motion blur from a single blurry image. We propose a deep learning approach to predicting the probabilistic distribution of motion blur at the patch level using a convolutional neural network (CNN). We further extend the candidate set of motion kernels predicted by the CNN using carefully designed image rotations. A Markov random field model is then used to infer a dense non-uniform motion blur field enforcing motion smoothness. Finally, motion blur is removed by a non-uniform deblurring model using patch-level image prior. Experimental evaluations show that our approach can effectively estimate and remove complex non-uniform motion blur that is not handled well by previous approaches.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/03eb5c9e91031b20f95ff9cf3f47013086fea9a2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Online Video Deblurring via Dynamic Temporal Blending Network (2017, background): “Previous learning-based single-image blind deconvolution [3, 28, 32] and burst deblurring [35] approaches have considered only ego motion and assumed a uniform blur model.”
Deep Video Deblurring for Hand-Held Cameras (2017, methodology): “CNNs have also been used for single- [2, 43] and multi[48] image deblurring, using synthetic training data.”
A Neural Approach to Blind Motion Deblurring (2016, background): “[15] considered a version of the problem with restrictions on motion blur types, and were able to successfully train a neural network to identify the blur in an observed natural image patch from among a small discrete set of oriented box blur kernels of various lengths.”
Learning to Super-Resolve Blurry Face and Text Images (2017, methodology): “Recently, neural networks have also been used for blind image deblurring [39, 3, 37].”
Deblurring via Stochastic Refinement (2021, methodology): “Alternatively, many recent methods adopt an end-to-end approach where a deep neural network is trained to directly produce a point estimate [8, 11, 14, 21, 35, 36, 50, 54, 55, 64, 66, 67, 73].”
Neural Blind Deconvolution Using Deep Priors (2019, methodology): “In [40], CNN is used to predict the parametric blur kernels for motion blurry images.”
Adversarial Spatio-Temporal Learning for Video Deblurring (2018, methodology): “There are also several deep learning methods that achieve encouraging results on deblurring [2], [46]–[51].”
Rethinking Coarse-to-Fine Approach in Single Image Deblurring (2021, background): “ods [30, 7, 1, 27] commonly exploit CNN as a blur kernel estimator and construct two-stage image deblurring framework, i.”
Deblurring by Realistic Blurring (2020, methodology): “For deep learning methods, Sun et al. [34] use a CNN network to estimate blur kernels and apply traditional deconvolution methods to synthesize sharp images.”
Unsupervised Domain-Specific Deblurring via Disentangled Representations (2019, background): “Although numerous blur generation methods have been developed [15, 38, 4], they are not capable of learning all types of blur variants in the wild.”
Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring (2016, methodology): “Recently, CNNs (Convolutional Neural Networks) have been applied in numerous computer vision problems and showed promising results including deblurringproblem [30, 25, 27, 1].”
Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model (2019, methodology): “Similar ideas have been employed in estimating the blur kernels in dynamic deblurring [44, 17] or convolutional kernels in video interpolation [36, 37].”
Deep Image Deblurring: A Survey (2022, methodology): “Networks using ResBlocks Nah et al. (2017) and DenseBlocks (Gao et al. 2019; Zhang et al. 2020) also achieve better performance than previous networks which directly stack convolutional layers (Gong et al. 2017; Sun et al. 2015).”
Deep Network Interpolation for Continuous Imagery Effect Transition (2018, background): “CNN-based approaches have led to a series of breakthroughs for several image restoration tasks including super-resolution [3, 18, 25, 19, 33, 20, 43], denoising [1, 44], de-blocking [40, 7] and deblurring [42, 32, 28].”
Video Frame Interpolation via Adaptive Separable Convolution (2017, methodology): “Our research is inspired by the success of applying deep learning to optical ﬂow estimation [2, 12, 16, 19, 50, 51, 52], artistic style transfer [17, 22, 28], and image enhancement [6, 9, 10, 46, 47, 55, 57, 62, 65].”
Lensless computational imaging through deep learning (2017, background): “…neural networks to “learn deeper;” and modern DNNs have shown spectacular success at solving “hard” computational problems, such as: playing complex games like Atari [17] and Go [18], object detection [19], and image restoration (e.g., colorization [20], deblurring [21–23], in-painting [24]).”
Human-Aware Motion Deblurring (2019, methodology): “Several early works [19, 38] directly convolved sharp images with a set of pre-defined motion kernels to synthesize blurry images.”