Return of the Devil in the Details: Delving Deep into Convolutional Nets

By Ken Chatfield, Karen Simonyan, Andrea Vedaldi, Andrew Zisserman. 2014

3,496 citations recorded by the source corpus.

The latest generation of Convolutional Neural Networks (CNN) have achieved impressive results in challenging benchmarks on image recognition and object detection, significantly raising the interest of the community in these methods. Nevertheless, it is still unclear how different CNN methods compare with each other and with previous state-of-the-art shallow representations such as the Bag-of-Visual-Words and the Improved Fisher Vector. This paper conducts a rigorous evaluation of these new techniques, exploring different deep architectures and comparing them on a common ground, identifying and disclosing important implementation details. We identify several useful properties of CNN-based representations, including the fact that the dimensionality of the CNN output layer can be reduced significantly without having an adverse effect on performance. We also identify aspects of deep and shallow methods that can be successfully shared. In particular, we show that the data augmentation techniques commonly applied to CNN-based methods can also be applied to shallow methods, and result in an analogous performance boost. Source code and models to reproduce the experiments in the paper is made publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fast R-CNN (2015, background): “” The second network is VGG CNN M 1024 from [3], which has the same depth as S, but is wider.”
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, background): “, increased depth [29, 33], enlarged width [37, 28], and the use of smaller strides [37, 28, 2, 29]), new nonlinear activations [24, 23, 38, 22, 31, 10], and sophisticated layer designs [33, 12].”
Convolutional neural networks at constrained time cost (2014, methodology): “The recent models [1, 9, 22, 23] take a high-end GPU or multiple GPUs/clusters one week or several weeks to train, which can sometimes be too demanding for the rapidly changing industry.”
Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, methodology): “On Caltech-256, our features outperform the state of the art [1] by a large margin ( 8.”
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014, methodology): “Next we describe our training solution that takes advantage of these GPU implementations while still preserving the spatial pyramid pooling behaviors.”
Do Better ImageNet Models Transfer Better? (2018, methodology): “Network architectures measured against this dataset have fueled much progress in computer vision research across a broad array of problems, including transferring to new datasets [2, 3], object detection [4], image segmentation [5, 6] and perceptual metrics of images [7].”
Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “by either independently classifying features extracted locally from object proposals [55,51,29] or by considering global image features and ﬁnetuning the pretrained models with a binary logistic loss [5,4,61]. By considering object proposals separately, the former approaches fail to consider potenti”
Training Region-Based Object Detectors with Online Hard Example Mining (2016, methodology): “We conduct experiments with two standard ConvNet architectures: VGG CNN M 1024 (VGGM, for short) from [5], which is a wider version of AlexNet [19], and VGG16 from [28].”
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, methodology): “We conduct extensive experiments evaluating choices like the number of layers combined, using a segmentation loss, normalizing feature amplitudes, different IRNN architectures, and other variations.”
Unsupervised Learning from Narrated Instruction Videos (2015, methodology): “rames into a single bag-of-visual-word vector of 2,000 dimensions [24]. The visual vocabulary is generated by k-means on a large set of training descriptors. To represent appearance, we use VGG-s CNN [6] and obtain 4,096-dimensional descriptors for each video frame corresponding to the “full7” layer”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, background): “Fine-tuning is performed end-to-end (all parameters are modified) which typically results in better accuracy than training only the final linear layer (Kornblith et al., 2019; Agrawal et al., 2014; Chatfield et al., 2014; Azizpour et al., 2015).”
Delving into Egocentric Actions (2015, positive): “Implementation Details: We also implement spatial FV (SFV) [17] and data augmentation [2].”
Two-Stream Convolutional Networks for Action Recognition in Videos (2014, background): “It corresponds to CNN-M-2048 architecture of [3] and is similar to the network of [31].”
Regularized Evolution for Image Classifier Architecture Search (2018, background): “Until recently, most state-of-the-art (SOTA) convolutional architectures have been manually designed by human experts (Krizhevsky et al., 2012; Ciregan et al., 2012; Lin et al., 2014; Sermanet et al., 2014; He et al., 2014; Chatfield et al., 2014; Simonyan & Zisserman, 2015; Karpathy et al., 2014; S”
Feedforward semantic segmentation with zoom-out features (2014, methodology): “ature dimensions. 4.2.3 Distant and global features For distant and global features we use deep convnets originally trained to classify images. In our initial experiments we used the CNN-S network in [7]. It has 5 convolution layers (three of them followed by pooling) and two fully connected layers.”
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding (2016, methodology): “We use the VGG-M model architecture [37] for both networks and follow the training procedure introduced in Simonyan et al.”
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (2015, background): “With the continuously improving deep convolutional models [31, 25, 3, 13, 26, 27, 14], the accuracy of these vision tasks also improves thanks to the more representative generic features learned from large-scale datasets.”
Quantized Convolutional Neural Networks for Mobile Devices (2015, methodology): “We demonstrate our approach on four convolutional networks: AlexNet [16], CaffeNet [15], CNN-S [1], and VGG16 [27].”