Manipulating Attributes of Natural Scenes via Hallucination

By Levent Karacan, Zeynep Akata, Aykut Erdem, Erkut Erdem. 2018

24 citations recorded by the source corpus.

In this study, we explore building a two-stage framework for enabling users to directly manipulate high-level attributes of a natural scene. The key to our approach is a deep generative network that can hallucinate images of a scene as if they were taken in a different season (e.g., during winter), weather condition (e.g., on a cloudy day), or at a different time of the day (e.g., at sunset). Once the scene is hallucinated with the given attributes, the corresponding look is then transferred to the input image while preserving the semantic details intact, giving a photo-realistic manipulation result. As the proposed framework hallucinates what the scene will look like, it does not require any reference style image as commonly utilized in most of the appearance or style transfer approaches. Moreover, it allows to simultaneously manipulate a given scene according to a diverse set of transient attributes within a single model, eliminating the need of training multiple networks per each translation task. Our comprehensive set of qualitative and quantitative results demonstrates the effectiveness of our approach against the competing methods.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic Image Synthesis With Spatially-Adaptive Normalization (2019, background): “An other widely-used form is image-to-image translation [16, 20,22,23,24,28,32,54,56,57], where both input and out put are images.”
Swapping Autoencoder for Deep Image Manipulation (2020, background): “The aim is to enable new class of editing operations, such as inpainting large image regions [65, 87, 58], synthesizing photorealistic images from layouts [35, 79, 64], replacing objects [95, 29], or changing the time photo is taken [43, 2].”