Semantic Image Synthesis With Spatially-Adaptive Normalization

By Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu. 2019

3,180 citations recorded by the source corpus.

We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the semantic layout as input to the network, forcing the network to memorize the information throughout all the layers. Instead, we propose using the input layout for modulating the activations in normalization layers through a spatially-adaptive, learned affine transformation. Experiments on several challenging datasets demonstrate the superiority of our method compared to existing approaches, regarding both visual fidelity and alignment with input layouts. Finally, our model allows users to easily control the style and content of image synthesis results as well as create multi-modal results. Code is available upon publication.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

  1. Andreas Geiger
  2. Jonathan T. Barron
  3. Eli Shechtman
  4. Ben Mildenhall
  5. Alexei A. Efros
  6. Daniel Cohen-Or
  7. Jun-Yan Zhu
  8. Pratul P. Srinivasan

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrea Vedaldi
  8. Andrew Zisserman

In Generative Models

  1. Jun-Yan Zhu
  2. Eli Shechtman
  3. Richard Zhang
  4. Daniel Cohen-Or
  5. Alexei A. Efros
  6. Stefano Ermon
  7. Bolei Zhou
  8. Ming-Hsuan Yang

Citations in the corpus