Visualizing and Understanding Deep Texture Representations

By Tsung-Yu Lin, Subhransu Maji. 2015

146 citations recorded by the source corpus.

A number of recent approaches have used deep convolutional neural networks (CNNs) to build texture representations. Nevertheless, it is still unclear how these models represent texture and invariances to categorical variations. This work conducts a systematic evaluation of recent CNN-based texture descriptors for recognition and attempts to understand the nature of invariances captured by these representations. First we show that the recently proposed bilinear CNN model [25] is an excellent generalpurpose texture descriptor and compares favorably to other CNN-based descriptors on various texture and scene recognition benchmarks. The model is translationally invariant and obtains better accuracy on the ImageNet dataset without requiring spatial jittering of data compared to corresponding models trained with spatial jittering. Based on recent work [13, 28] we propose a technique to visualize pre-images, providing a means for understanding categorical properties that are captured by these representations. Finally, we show preliminary results on how a unified parametric model of texture analysis and synthesis can be used for attribute-based image manipulation, e.g. to make an image more swirly, honeycombed, or knitted. The source code and additional visualizations are available at http://vis-www.cs.umass.edu/texture.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Do Better ImageNet Models Transfer Better? (2018, methodology): “Several papers have proposed methods to make better use of CNN features and thus improve the efficacy of transfer learning [43, 11, 42, 23, 73, 63, 13, 41, 54].”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “pute the gradients of the objective with respect to x using back-propagation. The hyperparameter = 10 8 was found empirically to lead to good inverse images. We also refer readers to our earlier work [36] and Birds VGG-D ﬁlters VGG-M ﬁlters Aircrafts VGG-D ﬁlters VGG-M ﬁlters Cars VGG-D ﬁlters VGG-M”
Visualizations of Deep Neural Networks in Computer Vision: A Survey (2017, methodology): “1, GTSRB trafc sign dataset, CIFAR10 Huang [28] 2012 Architecture assessment Pixel displays Representation Learning CDBN LFW Szegedy et al. [63] 2013 Architecture assessment Heat maps Detection DNN VOC2007 Long et al.[45] 2014 Architecture assessment Pixel displays Classification CNNs ImageNet, VOC ”
Texture Feature Extraction Methods: A Survey (2019, background): “They reported that the studied models are a good approach for texture synthesis and manipulation of content of images using texture attributes [247].”
Kernel Pooling for Convolutional Neural Networks (2017, background): “Representative examples include fine-grained visual recognition [23, 11], visual question answering [10], texture representation and synthesis [13, 22], face recognition [35] and style transfer [12].”
From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge (2019, result): “All participants extracted small image patches of metastases and normal areas from the WSIs to train their CNNs, although the exact patch size and pixel resolution differed substantially.”
Swapping Autoencoder for Deep Image Manipulation (2020, background): “[57] Lin, T.”
From BoW to CNN: Two Decades of Texture Representation for Texture Classification (2018, methodology): “, the finetuned FCCNN descriptor) [36, 120].”
Bilinear CNNs for Fine-grained Visual Recognition (2015, methodology): “We also refer readers to our earlier work [36] and Birds”
Locally-Transferred Fisher Vectors for Texture Classification (2017, methodology): “In addition, recently a compact bilinear pooling (CBP) method [9] was proposed to reduce the dimension of the B-CNN feature.”