Analyzing the Performance of Multilayer Neural Networks for Object Recognition

By Pulkit Agrawal, Ross B. Girshick, Jitendra Malik. 2014

465 citations recorded by the source corpus.

In the last two years, convolutional neural networks (CNNs) have achieved an impressive suite of results on standard recognition datasets and tasks. CNN-based features seem poised to quickly replace engineered representations, such as SIFT and HOG. However, compared to SIFT and HOG, we understand much less about the nature of the features learned by large CNNs. In this paper, we experimentally probe several aspects of CNN feature learning in an attempt to help practitioners gain useful, evidence-backed intuitions about how to apply CNNs to computer vision problems.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Object Detection Networks on Convolutional Feature Maps (2015, methodology): “We investigate two sets of training images: 162 (i) the original trainval set of 5k images in VOC 2007, 163 and (ii) an augmented set of 16k images that consists 164 of VOC 2007 trainval images and VOC 2012 trainval 165 images, following [27].”
Rethinking ImageNet Pre-Training (2018, neutral): “Given the success of pre-training in the R-CNN paper [9], later analysis [1] found that pre-training plays an important role in detector accuracy when training data is limited, but also illustrated that training from scratch on more detection data is possible and can achieve 90% of the fine-tuning a”
Do Better ImageNet Models Transfer Better? (2018, methodology): “A substantial body of existing research indicates that, in image tasks, fine-tuning typically achieves higher accuracy than classification based on fixed features, especially for larger datasets or datasets with a larger domain mismatch from the training set [1, 6, 24, 74, 2, 43, 33, 9, 51].”
Exploring the Limits of Weakly Supervised Pretraining (2018, background): “, [12,13]) or the use of different datasets that are of the same size magnitude as ImageNet (e.”
What makes ImageNet good for transfer learning? (2016, methodology): “For SUN-CLS we used the same train/val/test splits as used by [2].”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, methodology): “The performance gap between our implementation of the ﬁne-tuning baseline and that in [1, 15] is mainly due to different number of iterations: we used twice of the number of epochs in [1, 15] ( 30 epochs), leading to improved accuracy.”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “These visual representations have been shown to be effective on a wide range of computer vision tasks [1, 4, 14, 22, 29, 33, 36].”
Webly Supervised Learning of Convolutional Networks (2015, methodology): “As baselines we compare against R-CNN trained using CNN-Scratch features [1] (VOC-Scratch), R-CNN trained on ImageNet features without ﬁne-tuning (ImageNet-NFT), R-CNN trained on ImageNet features with ﬁne-tuning on VOC trainval (ImageNet-FT) and our webly trained CNN without ﬁne-tuning…”
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2016, neutral): “Visual tasks, such as object classification and detection, have been successfully solved through the supervised learning paradigm [1,10,22,31], where one uses labeled data to train a parametric model.”
Unsupervised Visual Representation Learning by Context Prediction (2015, methodology): “We do not use boundingbox regression, and take the appropriate results from [19] and [2].”
Instance-Sensitive Fully Convolutional Networks (2016, methodology): “6 we compare with state-of-the-art segment proposal methods: Selective Search (SS) [6], Multiscale Combinatorial Grouping (MCG) [12], ∼ DeepMask, and Multi-task Network Cascade (MNC) [20].”
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, methodology): “Since we know that pre-training on ImageNet is important to achieve state-of-the-art performance [1], and we would like to use the previously trained VGG16 network [37], it is important to preserve the existing layer shapes.”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, background): “Fine-tuning is performed end-to-end (all parameters are modiﬁed) which typically results in better accuracy than training only the ﬁnal linear layer (Kornblith et al., 2019; Agrawal et al., 2014; Chatﬁeld et al., 2014; Azizpour et al., 2015).”
Learning to See by Moving (2015, background): “…in computer vision have shown that visual features learnt by training neural networks for the task of object recognition using more than a million labelled images are useful for many computer vision tasks like semantic segmentation, object detection and action classiﬁ-cation [22, 13, 17, 1, 38].”
Object Detectors Emerge in Deep Scene CNNs (2014, background): “In Agrawal et al. (2014) and Szegedy et al. (2013), they suggest that the CNN for ImageNet learns a distributed code for objects.”
Learning without Forgetting (2016, background): “priate hyper-parameters for training, the resulting model often outperforms feature extraction [6], [13] or learning from a randomly initialized network [14], [15].”
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (2015, methodology): “In [1], it has been found that augmenting the object detection training set by combining the VOC 2007 and VOC 2012 sets improves object detection accuracy compared with using VOC 2007 only.”
Inverting Visual Representations with Convolutional Networks (2015, background): “Although it is known that binarized ConvNet features perform well in classification [1], it comes as a surprise that for reconstructing the input image the exact values of the features are not important.”
Network Dissection: Quantifying Interpretability of Deep Visual Representations (2017, background): “Although we expect a network to learn partially nonlocal representations in interior layers [5], and past experience shows that an emergent concept will often align with a combination of a several hidden units [11, 2], street (scene) flower (object) headboard (part)”