Image Inpainting via Conditional Texture and Structure Dual Generation

By Xiefan Guo, Hongyu Yang, Di Huang. IEEE International Conference on Computer Vision, 2021

298 citations recorded by the source corpus.

Deep generative approaches have recently made considerable progress in image inpainting by introducing structure priors. Due to the lack of proper interaction with image texture during structure reconstruction, however, current solutions are incompetent in handling the cases with large corruptions, and they generally suffer from distorted results. In this paper, we propose a novel two-stream network for image inpainting, which models the structure-constrained texture synthesis and texture-guided structure reconstruction in a coupled manner so that they better leverage each other for more plausible generation. Furthermore, to enhance the global consistency, a Bi-directional Gated Feature Fusion (Bi-GFF) module is designed to exchange and combine the structure and texture information and a Contextual Feature Aggregation (CFA) module is developed to refine the generated contents by region affinity learning and multi-scale feature aggregation. Qualitative and quantitative experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate the superiority of the proposed method. Our code is available at https://github.com/Xiefan-Guo/CTSDG.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/775113485783093d9cf246f4ef16fc40f61f20e4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Contrastive Learning for Diverse Disentangled Foreground Generation (2022, background): “While tremendous progress has been made to obtain better realism and coherence [62,11,42,26], progress in diversity is known-factor space (identity) input unkno w n -f ac t o r s p ace Fig.”
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (2022, unknown): “We compare our method with two state-of-the-art face inpainting methods of CTSDG [14] and GPEN [44] and give more results in Fig.”
RePaint: Inpainting using Denoising Diffusion Probabilistic Models (2022, background): “…[48] to guide the convolution kernel according to the inpainted mask, Contextual Attention [46] to leverage on global information, Edges maps [9,27,43,44] or Semantic Segmentation maps [14, 31] to further guide the generation, and Fourier Convolutions [40] to include both global and lo-cal…”