SinGAN: Learning a Generative Model From a Single Natural Image

By Tamar Rott Shaham, Tali Dekel, Tomer Michaeli. 2019

954 citations recorded by the source corpus.

We introduce SinGAN, an unconditional generative model that can be learned from a single natural image. Our model is trained to capture the internal distribution of patches within the image, and is then able to generate high quality, diverse samples that carry the same visual content as the image. SinGAN contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. This allows generating new samples of arbitrary size and aspect ratio, that have significant variability, yet maintain both the global structure and the fine textures of the training image. In contrast to previous single image GAN schemes, our approach is not limited to texture images, and is not conditional (i.e. it generates samples from noise). User studies confirm that the generated samples are commonly confused to be real images. We illustrate the utility of SinGAN in a wide range of image manipulation tasks.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Generative Adversarial Networks (2021, neutral): “By providing a thorough overview of GANs, we hope to make this complex and challenging topic more accessible to researchers and practitioners in the field, and inspire new innovations and applications.”
A critical analysis of self-supervision, or what we can learn from a single image (2019, background): “A single image can also be used to learn and interpolate multi-scale textures with a GAN framework (Rott Shaham et al., 2019).”
The Perception-Distortion Tradeoff (2017, background): “GAN-based image restoration Most recently, GAN-based methods have demonstrated unprecedented perceptual quality in super-resolution [3], [6], [9], [37], inpainting [7], [20], [38], compression [21], [39], [40], deblurring [41] and image-to-image translation [11], [22], [42].”
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models (2023, background): “In the past decade, we have witnessed the rapid advance of it from unconditional generative models to conditional generative models with powerful architectures of auto-regressive model [36, 50], GAN [4, 24, 25, 44, 47] and diffusion process [2, 14, 21, 30, 34, 45].”
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors (2023, background): “Furthermore, the animated objects are limited to specific categories, e.g ., fluid [31, 31, 45, 51], natural scenes [9, 36, 42, 60, 84], human hair [82], portraits [21, 78, 79], and bodies [4, 6, 37, 65, 79, 81].”
Positional Encoding as Spatial Inductive Bias in GANs (2020, methodology): “In addition to internal generative model SinGAN [42], we also study various unconditional generator architectures, including DCGAN [36], PGGAN [20], and StyleGAN2 [22].”
Swapping Autoencoder for Deep Image Manipulation (2020, methodology): “(Right) Using the self-similarity distance [45] and SIFID [66], we study variations of the co-occurrence discriminator’s patch size in training with respect to the image size.”
Remix: Rebalanced Mixup (2020, background): “Deep neural networks have made notable breakthroughs in many fields such as computer vision [35,27,20], natural language processing [18,19,9] and reinforcement learning [28].”
Meta-Transfer Learning for Zero-Shot Super-Resolution (2020, background): “Also, some works to learn internal distribution have been proposed [34, 32, 33].”
BlobGAN: Spatially Disentangled Scene Representations (2022, background): “This interaction is related to traditional “image reshuffling” where rearrangement of image content is done in pixel space [81, 4, 70].”
StyleFormer: Real-time Arbitrary Style Transfer via Parametric Style Composition (2021, background): “[1] discovered that the features extracted from a well-trained deep convolutional neural network can indicate the content structures, and their statistical distributions capture the style patterns, which inspired a line of works with advanced style and content descriptions [2, 3, 4, 5, 6, 7].”
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (2021, background): “…as generative adversarial networks (GAN) [3, 10, 19], normalizing flows [21], and variational autoencoders [42], have shown remarkable quality in image generation, and have been applied to numerous purposes such as image-to-image translation [7, 11, 31, 32, 35, 47] and image editing [1, 12, 36].”
Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches (2020, background): “Recently, progressive training strategy has been widely utilized for generation tasks [19,27,32,1], since it can simplify the information propagation within the network by intermediate supervision.”
Image-to-Image Translation: Methods and Applications (2021, positive): “A lower SIFID score indicates that the styles of two images are more similar.”
Few-Shot Adaptation of Generative Adversarial Networks (2020, background): “Recent work in one-shot image synthesis has demonstrated highquality and diverse results by modeling the internal distribution of features from a single image without pretraining (Shaham et al., 2019; Shocher et al., 2019).”