Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart

By Chao Yang, Xiaofeng Liu, Qingming Tang, C.-C. Jay Kuo. 2018

25 citations recorded by the source corpus.

Recent advances in deep generative models have shown promising potential in image inpanting, which refers to the task of predicting missing pixel values of an incomplete image using the known context. However, existing methods can be slow or generate unsatisfying results with easily detectable flaws. In addition, there is often perceivable discontinuity near the holes and require further post-processing to blend the results. We present a new approach to address the difficulty of training a very deep generative model to synthesize high-quality photo-realistic inpainting. Our model uses conditional generative adversarial networks (conditional GANs) as the backbone, and we introduce a novel block-wise procedural training scheme to stabilize the training while we increase the network depth. We also propose a new strategy called adversarial loss annealing to reduce the artifacts. We further describe several losses specifically designed for inpainting and show their effectiveness. Extensive experiments and user-study show that our approach outperforms existing methods in several tasks such as inpainting, face completion and image harmonization. Finally, we show our framework can be easily used as a tool for interactive guided inpainting, demonstrating its practical value to solve common real-world challenges.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Thomas S. Huang
2. Luc Van Gool
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Jiahui Yu
6. Trevor Darrell
7. Yi Yang
8. Xiaogang Wang

In Generative Models

1. Jiahui Yu
2. Thomas S. Huang
3. Zhe L. Lin
4. Jimei Yang
5. Xin Lu
6. Xiaohui Shen
7. Dacheng Tao
8. Jianlong Fu

Citations in the corpus

• Free-Form Image Inpainting With Gated Convolution (2018, background): “The latter approach [15, 49, 45, 46, 38, 37, 48, 26, 52, 33, 35, 19] can exploit semantics learned from large scale datasets to synthesize contents in nonstationary images in an end-to-end fashion.”
• Coherent Semantic Attention for Image Inpainting (2019, unknown): “Learning approaches [9, 15, 25, 33, 38, 40, 43] often use deep learning and GAN strategy to generate pixels of the hole.”