Texture and art with deep neural networks.

By Leon A. Gatys, Alexander S. Ecker, Matthias Bethge. 2017

84 citations recorded by the source corpus.

Although the study of biological vision and computer vision attempt to understand powerful visual information processing from different angles, they have a long history of informing each other. Recent advances in texture synthesis that were motivated by visual neuroscience have led to a substantial advance in image synthesis and manipulation in computer vision using convolutional neural networks (CNNs). Here, we review these recent advances and discuss how they can in turn inspire new research in visual perception and computational neuroscience.

Subfields: Biophysics, Cognitive Neuroscience, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

1. Qi Tian
2. Matthias Bethge
3. Richard S. Zemel
4. Felix Wichmann
5. Robert Geirhos
6. Lingxi Xie
7. Wieland Brendel
8. Jörn-Henrik Jacobsen

In Cognitive Neuroscience

1. Matthias Bethge
2. Robert Geirhos
3. Wieland Brendel
4. Felix Wichmann
5. Claudio Michaelis
6. Qi Tian
7. Richard S. Zemel
8. Lingxi Xie

In Computer Vision and Pattern Recognition

1. Qi Tian
2. Luc Van Gool
3. Alan L. Yuille
4. Trevor Darrell
5. Dacheng Tao
6. Yi Yang
7. Ming-Hsuan Yang
8. Bernt Schiele

In Generative Models

1. Wieland Brendel
2. Felix Wichmann
3. Robert Geirhos
4. Claudio Michaelis
5. Jens Behrmann
6. Matthias Bethge
7. Patricia Rubisch
8. Qi Tian

Citations in the corpus

• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness (2018, methodology): “For the more difficult case (e.g. man-made objects), we made use of the fact that every object can become a texture if it is used not in isolation, but rather in a clutter of many objects of the same kind (e.g. Gatys et al., 2017).”
• Excessive Invariance Causes Adversarial Vulnerability (2018, background): “It has indeed been shown that deep networks strongly rely on spectral statistical regularities (Jo & Bengio, 2017), or stationary statistics (Gatys et al., 2017) to make their decisions, rather than more abstract features like shape and appearance.”
• Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data (2019, background): “ds [36, 23], the objective function can not converge to the global minimum. The DNN tends to learn low-level features firstly which can be easier to learn for machines such as size, color, and texture [13, 14, 15]. Brendel and Bethge [4] found that the decision-making behavior of current neural netwo”