DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

By Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell. 2013

5,097 citations recorded by the source corpus.

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be repurposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks. We investigate and visualize the semantic clustering of deep convolutional features with respect to a variety of such tasks, including scene recognition, domain adaptation, and fine-grained recognition challenges. We compare the efficacy of relying on various network levels to define a fixed feature, and report novel results that significantly outperform the state-of-the-art on several important vision challenges. We are releasing DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Multi-task Sequence to Sequence Learning (2015, methodology): “In the context of deep neural networks, MTL has been applied successfully to various problems ranging from language (Liu et al., 2015), to vision (Donahue et al., 2014), and speech (Heigold et al., 2013; Huang et al., 2013).”
Universal Language Model Fine-tuning for Text Classification (2018, methodology): “approach has been superseded by fine-tuning either the last (Donahue et al., 2014) or several of the last layers of a pretrained model and leaving the remaining layers frozen (Long et al.”
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “ossible to train general-purpose feature extractors that, with no or minimal retraining, can extract useful features for a variety of styles of data (Krizhevsky et al., 2012; Zeiler and Fergus, 2014; Donahue et al., 2014). However, attempts to bring this kind of general purpose representation learni”
Fully convolutional networks for semantic segmentation (2014, methodology): “Transfer was first demonstrated on various visual recognition tasks [3, 38], then on detection, and on both instance and semantic segmentation in hybrid proposalclassifier models [10, 15, 13].”
Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, background): “n ILSVRC, as feature extractors on other, smaller, datasets, where training large models from scratch is not feasible due to over-ﬁtting. Recently, there has been a lot of interest in such a use case [1, 5, 17, 25], as it turns out that deep image representations, learnt on ILSVRC, generalise well t”
Caffe: Convolutional Architecture for Fast Feature Embedding (2014, methodology): “…CUDA code and GPU computation ﬁt industry needs by achieving processing speeds of more than 40 mil-a [7] unspeciﬁed C++ Python discontinued Decaf [2] BSD Python discontinued OverFeat [9] unspeciﬁed Lua C++,Python centralized Theano/Pylearn2 [4] BSD Python distributed Torch7 [1] BSD Lua…”
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014, background): “The output of the spatial pyramid pooling layer of this 180-network has the same fixed length as the 224-network.”
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation (2013, methodology): “ning, our system achieves a mAP of 54% on VOC 2010 compared to 33% for the highly-tuned, HOG-based deformable part model (DPM) [17,20]. We also point readers to contemporaneous work by Donahue et al. [12], who show that Krizhevsky’s CNN can be used (without ﬁnetuning) as a blackbox feature extractor”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, background): “In computer vision, models pretrained on large-scale image classiﬁcation have been shown to learn effective features for a range of problems (Donahue et al., 2014).”
Matching Networks for One Shot Learning (2016, methodology): “We then took this network and used the features from the last layer (before the softmax) for nearest neighbour matching, a strategy commonly used in computer vision [3] which has achieved excellent results across many tasks.”
Rethinking ImageNet Pre-Training (2018, background): “Deep convolutional neural networks [21, 23] revolutionized computer vision arguably due to the discovery that feature representations learned on a pre-training task can transfer useful information to target tasks [9, 6, 50].”
Do Better ImageNet Models Transfer Better? (2018, background): “No benchmark has been as hotly pursued as ImageNet [1].”
Exploring the Limits of Weakly Supervised Pretraining (2018, background): “This formula [1,2,3] has been in wide use for several years and led to impressive improvements on numerous tasks.”
What makes ImageNet good for transfer learning? (2016, methodology): “Using ImageNet pre-trained CNN features, impressive results have been obtained on several image classification datasets [13, 35], object detection [16, 39], action recognition [41], human pose estimation [9], image segmentation [10], optical flow [47], image captioning [12, 22] and others [26].”