A Neural Algorithm of Artistic Style

By Leon A. Gatys, Alexander S. Ecker, Matthias Bethge. 2015

3,091 citations recorded by the source corpus.

In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Learning to learn by gradient descent by gradient descent (2016, methodology): “Details can be found in [Gatys et al., 2015].”
Parallel WaveNet: Fast High-Fidelity Speech Synthesis (2017, methodology): “A similar principle has been used in computer vision for artistic style transfer [5], or to get better perceptual reconstruction losses, e.g., in super-resolution [12].”
Conditional Image Generation with PixelCNN Decoders (2016, background): “Neural artwork [18, 5] and content generation represent potential future uses for conditional generation.”
CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “An interesting result is that adjusting descriptive adjectives not only changes the textures of the drawings, akin to Style Transfer methods (Gatys et al., 2015), but also changes its structural representation of the underlying content.”
Generating Images with Perceptual Similarity Metrics based on Deep Networks (2016, methodology): “To control the degree of realism in generated images, an alternative to adversarial training is an approach making use of feature statistics, similar to (Gatys et al., 2015).”
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, positive): “Perceptually more convincing results were obtained for both super-resolution and artistic style-transfer [15, 16].”
Generating images with recurrent adversarial networks (2016, methodology): “A problem with generative adversarial models is that there is no obvious way to evaluate them quantitatively.”
InstructPix2Pix: Learning to Follow Image Editing Instructions (2022, neutral): “Generative models for image editing Image editing models traditionally targeted a single editing task such as style transfer [15, 16] or translation between image domains [22, 24, 36, 42, 71].”
Perceptual Losses for Real-Time Style Transfer and Super-Resolution (2016, methodology): “For style transfer our feed-forward networks are trained to solve the optimization problem from [11]; our results are similar to [11] both qualitatively and as measured by objective function value, but are three orders of magnitude faster to generate.”
Controllable Abstractive Summarization (2017, background): “Research in computer vision includes style transfer (Gatys et al., 2015) or controllable image generation (Lample et al., 2017).”
A Learned Representation For Artistic Style (2016, methodology): “The second point is motivated by the observation that the artistic style of a painting may be interpreted as a visual texture (Gatys et al., 2015a).”
Born Again Neural Networks (2018, positive): “…that requires the student to match the statistics of these inner products to the corresponding statistics calculated on the teacher (for the same example), by minimizing The authors exploit a statistic used in Gatys et al. (2015) to capture style similarity between images (given the same network).”
Neural Photo Editing with Introspective Adversarial Networks (2016, methodology): “Another related interface (Champanard, 2016) reﬁnes simple user input into complex textures through use of artistic style transfer (Gatys et al., 2015).”
Stacked Generative Adversarial Networks (2016, background): “In this section we will first introduce the adversarial loss LadvGi .”