A Layer-Based Sequential Framework for Scene Generation with GANs

By Mehmet Ozgur Turkoglu, William Thong, Luuk J. Spreeuwers, Berkay Kicanaoglu. 2019

38 citations recorded by the source corpus.

The visual world we sense, interpret and interact everyday is a complex composition of interleaved physical entities. Therefore, it is a very challenging task to generate vivid scenes of similar complexity using computers. In this work, we present a scene generation framework based on Generative Adversarial Networks (GANs) to sequentially compose a scene, breaking down the underlying problem into smaller ones. Different than the existing approaches, our framework offers an explicit control over the elements of a scene through separate background and foreground generators. Starting with an initially generated background, foreground objects then populate the scene one-by-one in a sequential manner. Via quantitative and qualitative experiments on a subset of the MS-COCO dataset, we show that our proposed framework produces not only more diverse images but also copes better with affine transformations and occlusion artifacts of foreground objects than its counterparts.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards causal generative scene models via competition of experts (2020, methodology): “Turkoglu et al. (2019) proposed a layer based model to add objects onto a background, Ashual & Wolf (2019) proposed a scene-generation method allowing for ﬁne grained user control, Karras et al. (2019a;b) have achieved impressive image generation results by exclusively training on a single class of…”
BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images (2020, background): “This includes conditional GANs that use segmentation masks [40, 49], scene graphs [22], object labels, key points or bounding boxes [18, 42], which have shown impressive results for natural image datasets.”
Adversarial Text-to-Image Synthesis: A Review (2021, methodology): “Another approach taken by [38] is to use instance masks for the desired objects and split the image generation process into foreground (objects), and background synthesis before blending them into the ﬁnal image (similar to [160] and iterative generation as in [161, 162, 93]).”
A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2021, background): “Since their development in 2014, generative adversarial training algorithms have been widely used in various unimodal applications such as scene generation [17], imageto-image translation [18], and image super-resolution [224, 225].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “[309], [310], [311], [312], [313], [314], [315], [316], to generate neural networks [317], to augment data [318], [319], to learn spatial representations [320], and in network embedding [321], heterogeneous information networks [322], and”