Perceptual Losses for Real-Time Style Transfer and Super-Resolution

By Justin Johnson, Alexandre Alahi, Li Fei-Fei. 2016

11,604 citations recorded by the source corpus.

We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a per-pixel loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing perceptual loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Xiaogang Wang
  5. Trevor Darrell
  6. Jitendra Malik
  7. Yi Yang
  8. Andrew Zisserman

In Generative Models

  1. Jun-Yan Zhu
  2. Eli Shechtman
  3. Richard Zhang
  4. Alexei A. Efros
  5. Stefano Ermon
  6. Daniel Cohen-Or
  7. Yoshua Bengio
  8. Ming-Hsuan Yang

Citations in the corpus