From generic to specific deep representations for visual recognition

By Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, Stefan Carlsson. 2014

433 citations recorded by the source corpus.

Evidence is mounting that ConvNets are the best representation learning method for recognition. In the common scenario, a ConvNet is trained on a large labeled dataset and the feed-forward units activation, at a certain layer of the network, is used as a generic representation of an input image. Recent studies have shown this form of representation to be astoundingly effective for a wide range of recognition tasks. This paper thoroughly investigates the transferability of such representations w.r.t. several factors. It includes parameters for training the network such as its architecture and parameters of feature extraction. We further show that different visual recognition tasks can be categorically ordered based on their distance from the source task. We then show interesting results indicating a clear correlation between the performance of tasks and their distance from the source task conditioned on proposed factors. Furthermore, by optimizing these factors, we achieve state-of-the-art performances on 16 visual recognition tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

What makes ImageNet good for transfer learning? (2016, background): “A thorough investigation of good architectural choices for transfer learning was conducted by [4].”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “, ImageNet), CNN features appear to transfer across a broad range of tasks [32, 4, 56].”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, background): “Fine-tuning is performed end-to-end (all parameters are modiﬁed) which typically results in better accuracy than training only the ﬁnal linear layer (Kornblith et al., 2019; Agrawal et al., 2014; Chatﬁeld et al., 2014; Azizpour et al., 2015).”
Fast ConvNets Using Group-Wise Brain Damage (2015, background): “Convolutional layers of large networks pretrained on large annotated training sets such as ILSVRC can be used as universal spatially localized features in a variety of ways [34, 1], which is particularly valuable for problems with considerably smaller training sets.”
Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks (2015, positive): “For ﬁne-grained classiﬁcation, we evaluate our approach on CUB200-2011 [35] (200 classes, 11788 images), NA birds [34] (555 classes, 48562 images), Stanford dogs [20] (120 classes, 20580 images), Oxford ﬂow-ers 102 [24] (102 classes, 8189 images), and Oxford-IIIT Pets [25] (37 classes, 7349 images).”
Particular object retrieval with integral max-pooling of CNN activations (2015, methodology): “No crop or change of aspect ratio is required (Azizpour et al., 2014).”
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters (2015, methodology): “applied DNNs to more than 10 different visual recognition challenge datasets, including human attribute prediction, fine-grained flower classification, and indoor scene recognition [3].”
The Devil is in the Tails: Fine-grained Classification in the Wild (2017, methodology): “This has become the standard method for obtaining baseline numbers on a new target dataset and often leads to impressive results [3], especially when the target dataset has sufﬁcient training examples.”
Ten Years of Pedestrian Detection, What Have We Learned? (2014, methodology): “No results have yet been reported using features pre-trained on ImageNet [54,57].”
Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification (2017, methodology): “[43], [47] demonstrate consistently using pre-trained CNN models can achieve better results via fine-tuning, which has been successfully used in some studies [38], [5], [34], [44].”
Fine-Tuning CNN Image Retrieval with No Human Annotation (2017, background): “Networks trained for image classification have shown strong adaptation abilities [3].”
Abnormality Detection and Localization in Chest X-Rays using Deep Convolutional Neural Networks (2017, result): “This observation is consistent with the conclusions drawn in [28].”
Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification (2016, background): “Many studies [3,10,31] have shown that fine-tuning a deep model pretrained on a large-scale dataset (e.”
LCNN: Lookup-Based Convolutional Neural Network (2016, background): “Few-Shot Learning: The problem of learning novel categories has been studied in [40, 2, 30].”
Evaluation of deep convolutional nets for document image classification and retrieval (2015, methodology): “Therefore, output extracted near the top of a CNN can serve as a feature vector which can be used for any task, including retrieval [29, 3, 14, 2].”