Semantic Image Inpainting with Perceptual and Contextual Losses

By Raymond A. Yeh, Teck-Yian Lim, Mark Hasegawa-Johnson, Minh N. Do. 2016

370 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, background): “However, the ability of MSE (and PSNR) to capture perceptually relevant differences, such as high texture detail, is very limited as they are defined based on pixel-wise image differences [58, 56, 25].”
A Hybrid Model for Identity Obfuscation by Face Replacement (2018, neutral): “We zero-pad the image if its dimensions are smaller than the crop size, thereby obtaining the ﬁnal crop with the desired 256 × 256 size.”
LOGAN: Membership Inference Attacks Against Generative Models (2017, neutral): “, compression [60], denoising [7], inpainting [69], superresolution [35], semi-supervised learning [54], etc.”
Generative Image Inpainting with Contextual Attention (2018, methodology): “It has many applications in photo editing, image-based rendering and computational photography [3, 23, 28, 29, 34, 39].”
LOGAN: Evaluating Privacy Leakage of Generative Models Using Generative Adversarial Networks (2017, background): “They are used in a plethora of applications, such as compression [40], denoising [4], inpainting [45], super-resolution [25], semisupervised learning [36], clustering [39], and deep neural networks pre-training [15] in cases where labeled data is expensive.”
Image Inpainting for Irregular Holes Using Partial Convolutions (2018, methodology): “[35], searches for the closest encoding to the corrupted image in a latent space, which is then used to condition the output of a hole-filling generator.”
Scribbler: Controlling Deep Image Synthesis with Sketch and Color (2016, background): “Conditional GANs Instead of synthesizing images from latent vectors, several works explore conditional GANs where the generator is conditioned on more meaningful inputs such as text [39, 38], low resolution images (superresolution) [24, 20], or incomplete images (inpainting) [35, 34, 51].”
TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network (2017, background): “Alternatively, Generative Adversarial Networks (GAN) [5] inception, having been used in tasks such as single-image super resolution [12], Simulated+Unsupervised learning [22], image-to-image translation [9] and semantic image inpainting [26].”
Fully differentiable full-atom protein backbone generation (2019, background): “Deep generative models, which harness the power of deep neural networks, have achieved remarkable results in realistic sample generation across many modalities including images (1; 2; 3; 4; 5), video (6; 7), audio (8), and symbolic expressions (9).”
Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision (2017, background): “Unsupervised inpainting networks [32] may not be necessary, since large supervised training datasets of paired masked and complete images can be collected via image masking.”
Compressed Sensing using Generative Models (2017, methodology): “The use of generative models for this task has been studied in (Yeh et al., 2016), where the objective is taken to be a sum of L”
Neural Architecture Search for Deep Image Prior (2020, background): “Generative Adversarial Networks (GANs) are more widely used for in-painting and super-resolution [43, 36] by learning structure and appearance from a representative corpus image data [41], in some cases explicitly maintaining both local and global consistency through independent models [21].”
Partial Convolution based Padding (2018, background): “Some other meth-ods [27, 31, 25] ignore the hole placeholder values in the image inpainting or data completion tasks; none of them has explicitly handled the missing data at the padded regions yet.”
Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart (2018, background): “In [28], Yeh et al. uses a pre-trained model to ﬁnd the most similar encoding of a corrupted image and uses the found encoding to synthesize what is missing.”
Neural Proximal Gradient Descent for Compressive Imaging (2018, background): “Deep neural networks have recently proven quite powerful in modeling prior distributions for images [9, 10, 11, 12, 13, 14].”
Coherent Semantic Attention for Image Inpainting (2019, methodology): “In contrast, early deep convolution neural networks based approaches [16, 22, 27, 26] learn data distribution to capture the semantic information of the image.”
Guided Image Inpainting: Replacing an Image Region by Pulling Content From Another Image (2018, background): “More recently, deep neural networks were introduced to regress from surrounding context to missing pixel values [28, 41, 18, 40, 42] or model the distribution of possible outputs by random sampling [44].”
Differentially Private Mixture of Generative Neural Networks (2017, background): “These models are used in a plethora of applications, such as compression [44], denoising [6], inpainting [52], super-resolution [28], semi-supervised learning [39], clustering [45], and deep neural networks pretraining [19] in cases where labeled data is expensive.”