Human-Aware Motion Deblurring

By Ziyi Shen, Wenguan Wang, Xiankai Lu, Jianbing Shen, Haibin Ling, Tingfa Xu, Ling Shao. IEEE International Conference on Computer Vision, 2019

392 citations recorded by the source corpus.

This paper proposes a human-aware deblurring model that disentangles the motion blur between foreground (FG) humans and background (BG). The proposed model is based on a triple-branch encoder-decoder architecture. The first two branches are learned for sharpening FG humans and BG details, respectively; while the third one produces global, harmonious results by comprehensively fusing multi-scale deblurring information from the two domains. The proposed model is further endowed with a supervised, human-aware attention mechanism in an end-to-end fashion. It learns a soft mask that encodes FG human information and explicitly drives the FG/BG decoder-branches to focus on their specific domains. Above designs lead to a fully differentiable motion deblurring network, which can be trained end-to-end. To further benefit the research towards Human-aware Image Deblurring, we introduce a large-scale dataset, named HIDE, which consists of 8,422 blurry and sharp image pairs with 65,784 densely annotated FG human bounding boxes. HIDE is specifically built to span a broad range of scenes, human object sizes, motion patterns, and background complexities. Extensive experiments on public benchmarks and our dataset demonstrate that our model performs favorably against the state-of-the-art motion deblurring methods, especially in capturing semantic details.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/20a4b3353579525f0b76ec42e17a2284b4453f9a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deblurring by Realistic Blurring (2020, methodology): “Shen et al. [30] introduce a human-aware deblurring method to remove blur from foreground humans and background.”
Uformer: A General U-Shaped Transformer for Image Restoration (2021, methodology): “Then we apply our GoPro trained model directly on the HIDE dataset [52] and the RealBlur dataset [48] to evaluate the generalization on real scenes.”
Rethinking Coarse-to-Fine Approach in Single Image Deblurring (2021, methodology): “We compared MIMO-UNet with state-of-the-art deblurring networks [20, 31, 5, 35, 29, 22, 23, 33, 28].”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “For image deblurring, similar to [36, 55, 63, 79, 80], we train our model with 2, 103 image pairs from GoPro [47] dataset and evaluate each method on the test sets from GoPro and HIDE [58], which contain 1, 111 and 2, 025 samples, respectively.”
DiffIR: Efficient Diffusion Model for Image Restoration (2023, methodology): “Method GoPro [41] HIDE [48] PSNR ↑ SSIM ↑ PSNR ↑ SSIM ↑”
Deep Image Deblurring: A Survey (2022, methodology): “The results on Shen et al.’s dataset Shen et al. (2019) demonstrate the effectiveness.”
MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “Deblurring GoPro [60] 2103 1111 GoPro HIDE [79] 0 2025 HIDE RealBlur-J [77] 3758 980 RealBlur-J RealBlur-R [77] 3758 980 RealBlur-R REDS [61] 24000 300 REDS”
Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring (2020, methodology): “Visual comparisons of deblurring results on images from the HIDE test set [18].”
Deblurring via Stochastic Refinement (2021, methodology): “We additionally evaluate our GoPro-trained model on the HIDE [61] dataset, which contains 2025 images also of size 1280 × 720 .”
Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring (2022, background): “Deblurred results on the HIDE dataset [9].”
Stripformer: Strip Transformer for Fast Image Deblurring (2022, methodology): “The HIDE dataset [34] with 2, 025 images is included only for testing.”
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration (2023, background): “On the HIDE dataset, the PSNR improvement is 0.43 dB.”
Multi-Stage Progressive Image Restoration (2021, methodology): “GoPro [53] HIDE [69] Method PSNR ↑ SSIM ↑ PSNR ↑ SSIM ↑”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, methodology): “Our Restormer is trained only on the GoPro dataset [56] and directly applied to the HIDE [67] and RealBlur [65] benchmark datasets.”