InOut: Diverse Image Outpainting via GAN Inversion

By Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Ming-Hsuan Yang. 2021

100 citations recorded by the source corpus.

Image outpainting seeks for a semantically consistent extension of the input image beyond its available content. Compared to inpainting - filling in missing pixels in a way coherent with the neighboring pixels - outpainting can be achieved in more diverse ways since the problem is less constrained by the surrounding pixels. Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the content available in the input image. In this work, we formulate the problem from the perspective of inverting generative adversarial networks. Our generator renders micro-patches conditioned on their joint latent code as well as their individual positions in the image. To outpaint an image, we seek for multiple latent codes not only recovering available patches but also synthesizing diverse outpainting by patch-based generation. This leads to richer structure and content in the outpainted regions. Furthermore, our formulation allows for outpainting conditioned on the categorical input, thereby enabling flexible user controls. Extensive experimental results demonstrate the proposed method performs favorably against existing in- and outpainting methods, featuring higher visual quality and diversity.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

GAN Inversion: A Survey (2021, neutral): “[142] propose a GAN inversion-basedmethod for image”
MaskGIT: Masked Generative Image Transformer (2022, methodology): “We evaluate on the image set generously provided by the authors of InfinityGAN [31] and In&Out [8].”
Palette: Image-to-Image Diffusion Models (2021, methodology): “2019], but are often domain-specific [Bowen et al. 2021; Cheng et al. 2021; Lin et al. 2021; Wang et al. 2019a; Yang et al. 2019a].”
Regularizing Generative Adversarial Networks under Limited Data (2021, unknown): “The GAN models are the corner-stone techniques for numerous vision applications, such as data augmentation [12, 13], domain adaptation [19, 20], image extrapolation [10, 66], image-to-image translation [22, 37, 86], and visual content creation [1, 4, 21, 23, 70, 72].”
LFS-GAN: Lifelong Few-Shot Image Generation (2023, background): “To enhance diversity within the target domain, the mode seeking loss [31] and its variants [7] have shown diverse image generation.”
InfinityGAN: Towards Infinite-Resolution Image Synthesis (2021, methodology): “We leverage the pipeline proposed in In&Out [4] to perform image outpainting with latent variable inversion.”