Recurrent Feature Reasoning for Image Inpainting

By Jingyuan Li, Ning Wang, Lefei Zhang, Bo Du, Dacheng Tao. 2020

445 citations recorded by the source corpus.

Existing inpainting methods have achieved promising performance for recovering regular or small image defects. However, filling in large continuous holes remains difficult due to the lack of constraints for the hole center. In this paper, we devise a Recurrent Feature Reasoning (RFR) network which is mainly constructed by a plug-and-play Recurrent Feature Reasoning module and a Knowledge Consistent Attention (KCA) module. Analogous to how humans solve puzzles (i.e., first solve the easier parts and then use the results as additional information to solve difficult parts), the RFR module recurrently infers the hole boundaries of the convolutional feature maps and then uses them as clues for further inference. The module progressively strengthens the constraints for the hole center and the results become explicit. To capture information from distant places in the feature map for RFR, we further develop KCA and incorporate it in RFR. Empirically, we first compare the proposed RFR-Net with existing backbones, demonstrating that RFR-Net is more efficient (e.g., a 4% SSIM improvement for the same model size). We then place the network in the context of the current state-of-the-art, where it exhibits improved performance. The corresponding source code is available at: https://github.com/jingyuanli001/RFR-Inpainting

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Resolution-robust Large Mask Inpainting with Fourier Convolutions (2021, methodology): “Inpainting frameworks often incorporate style [26, 30, 30, 47, 32, 25] and feature matching [51, 44, 32, 16] losses.”
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, methodology): “Our model is compared against RFR (Li et al., 2020) and DeepFillv2 (Yu et al., 2019), the state-of-the-art algorithms for free-form image completion, using both their official pre-trained models and our retrained version of DeepFillv2 (using the official code, our datasets, and our sampling strategy”
MAT: Mask-Aware Transformer for Large Hole Image Inpainting (2022, background): “To allow for high-resolution image inpainting, a few attempts have been made to develop progressively generation systems, such as [18, 32, 41, 71, 72].”
Image Inpainting via Conditional Texture and Structure Dual Generation (2021, methodology): “With the Recurrent Feature Reasoning module, RFR [11] yields competitive results; however, the details are still not so elegant as ours (the face and sky in Figure 5 serve as examples).”
PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization (2021, methodology): “For removal, inspired by [34], [67], we adopt the SOTA inpainting method [6] to fill one annotated region that is randomly removed from each chosen MS COCO image.”