Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation

By Yuqing Ma, Xianglong Liu, Shihao Bai, Lei Wang, Dailan He, Aishan Liu. 2019

64 citations recorded by the source corpus.

Recently deep neural networks have achieved promising performance for filling large missing regions in image inpainting tasks. They usually adopted the standard convolutional architecture over the corrupted image, where the same convolution filters try to restore the diverse information on both existing and missing regions, and meanwhile ignores the long-distance correlation among the regions. Only relying on the surrounding areas inevitably leads to meaningless contents and artifacts, such as color discrepancy and blur. To address these problems, we first propose region-wise convolutions to locally deal with the different types of regions, which can help exactly reconstruct existing regions and roughly infer the missing ones from existing regions at the same time. Then, a non-local operation is introduced to globally model the correlation among different regions, promising visual consistency between missing and existing regions. Finally, we integrate the region-wise convolutions and non-local correlation in a coarse-to-fine framework to restore semantically reasonable and visually realistic images. Extensive experiments on three widely-used datasets for image inpainting tasks have been conducted, and both qualitative and quantitative experimental results demonstrate that the proposed model significantly outperforms the state-of-the-art approaches, especially for the large irregular missing regions.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Regionwise Generative Adversarial Image Inpainting for Large Missing Areas (2019, methodology): “Note that we extend upon our prior conference publication [14] that mainly concentrated on the discontiguous missing areas using region-wise convolutions and suffered severe artifacts when the large missing areas are contiguous.”
Text-Guided Neural Image Inpainting (2020, methodology): “Following common inpainting evaluation setting [19, 20, 41, 44], a square mask taking 50% area is constructed at the center of every image.”
Image Inpainting: A Review (2019, methodology): “For example in [66] the PSNR value become 22.”