Visualizing and Understanding Convolutional Networks
By Matthew D. Zeiler, Rob Fergus. 2013
17,221 citations recorded by the source corpus.
Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark Krizhevsky et al. [18]. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we explore both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.
Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Jianfeng Gao
- Sergey Levine
- Trevor Darrell
- Dacheng Tao
- Percy Liang
In Biophysics
- Wojciech Samek
- K. Müller
- Andrea Vedaldi
- Grgoire Montavon
- Lior Wolf
- Antonio Torralba
- Bolei Zhou
- Been Kim
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Yi Yang
In Safety, Robustness & Interpretability
- Wojciech Samek
- K. Müller
- Been Kim
- Nicholas Carlini
- Matthias Hein
- Nicolas Papernot
- Longo Luca
- Giulia Vilone
Citations in the corpus
- Xception: Deep Learning with Depthwise Separable Convolutions (2016, background): “What followed was a trend to make this style of network increasingly deeper, mostly driven by the yearly ILSVRC competition; first with Zeiler and Fergus in 2013 [25] and then with the VGG architecture in 2014 [18].”
- Dissecting Contextual Word Embeddings: Architecture and Representation (2018, background): “1500 trained for image classification learn a hierarchy of image features (Zeiler and Fergus, 2014).”
- A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “…areas outside NLU, artificial neural network techniques have made it possible to train general-purpose feature extractors that, with no or minimal retraining, can extract useful features for a variety of styles of data (Krizhevsky et al., 2012; Zeiler and Fergus, 2014; Donahue et al., 2014).”
- How transferable are features in deep neural networks? (2014, methodology): “For example, Zeiler and Fergus (2013) found that it is better to decrease the first layer filters sizes from 11 × 11 to 7 × 7 and to use a smaller stride of 2 instead of 4.”
- Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, neutral): “Besides object representation, our work is also motivated by the observation that, due to the remarkable classification capability of the recent CNN models [17, 30, 25, 16, 13, 26], the bottleneck for good detection performance is now the accurate object localization.”
- Fully convolutional networks for semantic segmentation (2014, background): “Transfer was first demonstrated on various visual recognition tasks [3, 38], then on detection, and on both instance and semantic segmentation in hybrid proposalclassifier models [10, 15, 13].”
- Deeply-Supervised Nets (2014, background): “In addition to the present observation that learned features in CNN are not always intuitive and discriminative [29], broader discussion of difficulties in training deep neural networks has appeared in [8, 19].”
- Going deeper with convolutions (2014, methodology): “No new data sources were used, for example, by the top entries in the ILSVRC 2014 competition besides the classification dataset of the same competition for detection purposes.”
- Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, methodology): “Our ConvNet configurations are quite different from the ones used in the top-performing entries to ILSVRC-2012 and ILSVRC-2013 competitions [13, 20, 25].”
- ImageNet Large Scale Visual Recognition Challenge (2014, methodology): “The network architectures were chosen using the visualization technique of (Zeiler and Fergus, 2013), and they were trained on the GPU following (Zeiler et al., 2011) using the dropout technique (Krizhevsky et al., 2012).”
- On the Number of Linear Regions of Deep Neural Networks (2014, methodology): “This visualization technique, a byproduct of our theoretical analysis, is similar to the one proposed by Zeiler and Fergus (2013), but is motivated by a different perspective.”
- Do Better ImageNet Models Transfer Better? (2018, methodology): “Soon after, it became clear that intermediate representations learned from ImageNet also provided substantial gains over hand-engineered features when transferred to other tasks [2, 3].”
- What makes ImageNet good for transfer learning? (2016, methodology): “Understanding internal representations of CNN: For understanding what information is encoded by different CNN features, several works have developed feature visualization methods using either deconvolution [50], backpropagation[40, 52] or by training a decoder network [14].”
- Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “…to a novel target task ( e.g ., SUN-397 scene classification) in two ways: (1) going deeper by adding more layers (top) and (2) going wider by adding more channels per layer (bottom). knowledge from a generative to a discriminative model, and has since been generalized with great success [10, 57].”
- Local Relation Networks for Image Recognition (2019, background): “These are combined at middle layers to form object parts, and then finally at higher layers, whole objects are represented [35].”