PEPSI : Fast Image Inpainting With Parallel Decoding Network

By Min-Cheol Sagong, Yong-Goo Shin, Seung-Wook Kim, Seung Park, Sung-Jea Ko. 2019

123 citations recorded by the source corpus.

Recently, a generative adversarial network (GAN)-based method employing the coarse-to-fine network with the contextual attention module (CAM) has shown outstanding results in image inpainting. However, this method requires numerous computational resources due to its two-stage process for feature encoding. To solve this problem, in this paper, we present a novel network structure, called PEPSI: parallel extended-decoder path for semantic inpainting. PEPSI can reduce the number of convolution operations by adopting a structure consisting of a single shared encoding network and a parallel decoding network with coarse and inpainting paths. The coarse path produces a preliminary inpainting result with which the encoding network is trained to predict features for the CAM. At the same time, the inpainting path creates a higher-quality inpainting result using refined features reconstructed by the CAM. PEPSI not only reduces the number of convolution operation almost by half as compared to the conventional coarse-to-fine networks but also exhibits superior performance to other models in terms of testing time and qualitative scores.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Resolution-robust Large Mask Inpainting with Fourier Convolutions (2021, background): “Some studies [40, 42] modify the framework so that coarse and fine result components are obtained simultaneously rather than sequentially.”
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks (2021, background): “Numerous follow-up works focus on the semantic context and texture, edges and contours, or hand-engineered architectures (Altinel et al., 2018; Ding et al., 2018; Iizuka et al., 2017; Jiang et al., 2019; Jo & Park, 2019; Lahiri et al., 2017; Liu et al., 2019a; Nazeri et al., 2019; Ren et al., 2019; ”
Free-Form Image Inpainting With Gated Convolution (2018, background): “The latter approach [15, 49, 45, 46, 38, 37, 48, 26, 52, 33, 35, 19] can exploit semantics learned from large scale datasets to synthesize contents in nonstationary images in an end-to-end fashion.”
Aggregated Contextual Transformations for High-Resolution Image Inpainting (2021, background): “The other kind of works stack serialized dilated convolution layers to capture distant contexts [13], [20], [44].”
Image Fine-grained Inpainting (2020, unknown): “3 ‐ 4 C n v ‐ n c a n v ‐ Similar to [29, 30], Song et al .”
Diversity-Generated Image Inpainting with Style Extraction (2019, background): “For this reason, Sagong et al. [21] proposed a structure (Pepsi) composed of a single shared coding network and a parallel decoding network with rough and patching paths, which can reduce the number of convolution operations.”
Learning Oracle Attention for High-Fidelity Face Completion (2020, background): “In order to generate visually realistic images, existing approaches typically adopt the non-local schemes that make use of contextual relations to fill in the missing pixels [32, 21, 39, 24].”
Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness (2021, background): “There are also frameworks that focus on avoiding the complexity of inpainting model by parallel decoding network [40] or concentrate on generating”
Contrastive Learning for Diverse Disentangled Foreground Generation (2022, background): “Recently, deep learning methods [8,18,33,35,48,50,52,53,55,59,28,57,61] dramatically boosted the quality, in terms of both visual quality and semantic coherence.”
PEPSI++: Fast and Lightweight Network for Image Inpainting (2019, methodology): “In the previous work [20], we introduced a novel network structure called parallel extended-decoder path for semantic inpainting (PEPSI), which aims at reducing the number of convolution operations as well as improving the inpainting performance.”