Learning Texture Manifolds with the Periodic Spatial GAN

By Urs M. Bergmann, Nikolay Jetchev, Roland Vollgraf. 2017

179 citations recorded by the source corpus.

This paper introduces a novel approach to texture synthesis based on generative adversarial networks (GAN) (Goodfellow et al., 2014). We extend the structure of the input noise distribution by constructing tensors with different types of dimensions. We call this technique Periodic Spatial GAN (PSGAN). The PSGAN has several novel abilities which surpass the current state of the art in texture synthesis. First, we can learn multiple textures from datasets of one or more complex large images. Second, we show that the image generation with PSGANs has properties of a texture manifold: we can smoothly interpolate between samples in the structured noise space and generate novel samples, which lie perceptually between the textures of the original dataset. In addition, we can also accurately learn periodical textures. We make multiple experiments which show that PSGANs can flexibly handle diverse texture and image data sources. Our method is highly scalable and it can generate output images of arbitrary large size.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, background): “As the follow-up work of SGAN, Bergmann et al. [55] proposed a Periodic Spatial GAN (PSGAN), which can learn periodic textures from a single image or complex large dataset.”
SinGAN: Learning a Generative Model From a Single Natural Image (2019, background): “Unconditional single image GANs have been explored only in the context of texture generation [3, 27, 31].”
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, background): “GANs [31, 52, 10, 32, 38, 19] have shown improved photorealism in image synthesis [40, 15, 69], translation [36, 70, 71], or manipulation [9, 60, 61].”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, background): “ realistic details, in contrast to only low-frequency accuracy from other hand-crafted metrics which average over training dataset. Therefore, GANs have shown improved photorealism in image synthesis [17,54,92], translation [45,93,94], or manipulation [8,79,80]. On the other hand, because of the nat”
Learning a Neural 3D Texture Space From 2D Exemplars (2019, background): “Some work has used adversarial training to capture the essence of textures [21, 3], including the non-stationary case [28] or even inside a single image [21].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “Field Subfield Method Super-resolution SRGAN [63], ESRGAN [64], Cycle-in-Cycle GANs [65], SRDGAN [66], TGAN [67] DR-GAN [68], TP-GAN [69], PG2 [70], PSGAN [71], Image synthesis and manipulation APDrawingGAN [72], IGAN [73], Image processing and computer vision introspective adversarial networks [74]”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, methodology): “AN [95], DMTGAN [96] Face landmark detection: SAN [97], Expose-GAN [98] Image super-resolution: SR-GAN [99], ESR-GAN [100], SR-DGAN [101], T-GAN [102] Texture synthesis: MGAN [103], SGAN [105], PSGAN [106] Sketch synthesis: TGAN [107], Sketchy-GAN [110], CA-GAN [111] Image-to-image translation: Pix2”
TextureGAN: Controlling Deep Image Synthesis with Texture Patches (2017, background): “[2] apply adversarial training at the pixel level to encourage the generated results to be indistinguishable from real texture.”