Generative Image Inpainting with Contextual Attention

By Jiahui Yu, Zhe L. Lin, Jimei Yang, Xiaohui Shen, Xin Lu, Thomas S. Huang. 2018

2,556 citations recorded by the source corpus.

Recent deep learning based approaches have shown promising results for the challenging task of inpainting large missing regions in an image. These methods can generate visually plausible image structures and textures, but often create distorted structures or blurry textures inconsistent with surrounding areas. This is mainly due to ineffectiveness of convolutional neural networks in explicitly borrowing or copying information from distant spatial locations. On the other hand, traditional texture and patch synthesis approaches are particularly suitable when it needs to borrow textures from the surrounding regions. Motivated by these observations, we propose a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions. The model is a feedforward, fully convolutional neural network which can process images with multiple holes at arbitrary locations and with variable sizes during the test time. Experiments on multiple datasets including faces (CelebA, CelebA-HQ), textures (DTD) and natural images (ImageNet, Places2) demonstrate that our proposed approach generates higher-quality inpainting results than existing ones. Code, demo and models are available at: https://github.com/JiahuiYu/generative_inpainting.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

  1. Eli Shechtman
  2. Jun-Yan Zhu
  3. Jonathan T. Barron
  4. Richard Zhang
  5. Alexei A. Efros
  6. Jia-Bin Huang
  7. Antonio Torralba
  8. Peter Wonka

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Xiaogang Wang
  5. Trevor Darrell
  6. Chen Change Loy
  7. Jitendra Malik
  8. Yi Yang

In Generative Models

  1. Jun-Yan Zhu
  2. Eli Shechtman
  3. Richard Zhang
  4. Alexei A. Efros
  5. Ming-Hsuan Yang
  6. Bolei Zhou
  7. Daniel Cohen-Or
  8. Stefano Ermon

Citations in the corpus