Factors of Transferability for a Generic ConvNet Representation

By Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, Stefan Carlsson. 2014

355 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Do Better ImageNet Models Transfer Better? (2018, background): “Similar work concluded that deeper networks produce higher accuracy across many transfer tasks, but wider networks produce lower accuracy [2].”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, methodology): “While there is a large body of work on transfer learning, much of it assumes a ﬁxed capacity model [32, 3, 6, 58, 15].”
Learning without Forgetting (2016, methodology): “ces365 [13], and adapting the network to CUB leads to a Places365 accuracy loss of 8:4% (3:8%þ 4:6%) for fine-tuning, 3.”
Domain Adaptive Transfer Learning with Specialist Models (2018, methodology): “yers in a network were highly coadapted. They also showed that distance between tasks matters for transfer learning and examined two different subsets (man-made v.s. natural objects). Azizpour et al. [3] also examined different factors of model design such as depth, width, data diversity and density”
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning (2018, methodology): “[4] conducted a useful study on a list of transfer learning tasks that have different similarity with the original ImageNet classification task (e.”
End-to-End Learning of Deep Visual Representations for Image Retrieval (2016, methodology): “The best methods in the literature (Tolias and Jégou 2015; Azizpour et al. 2015) are hardly scalable as they require a lot of storage memory and an expensive verification.”
Interventional Few-Shot Learning (2020, background): “Many research works have being focused on when and how to conduct this transfer learning [28, 4, 76].”
SpotTune: Transfer Learning Through Adaptive Fine-Tuning (2018, background): “Alternatively, the last few layers of the deep network can be fine-tuned while freezing the parameters of the remaining initial layers to their pre-trained values [44, 1].”
Transferring GANs: generating images from limited data (2018, methodology): “Either by using these networks as off-the-shelf feature extractors [3], or by adapting them to a new domain by a process called fine tuning [33].”
Domain-Adaptive Deep Network Compression (2017, background): “The amount of compression seems to correlate with the similarity [3, 36] between the source and target domain when we compare it to the ordering proposed in [3] (see Table II).”
Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks (2016, background): “Instead of learning CNNs from scratch, many CNNs have recently been based on the idea that CNN features are quite general and that CNNs can be pre-trained on large, often unrelated, datasets and can then be fine-tuned on the data at hand [4].”
Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery (2021, methodology): “The ﬁrst method is feature extraction [63], which freezes the task-shared layers and task-speciﬁc layers of old tasks and only optimizes the speciﬁc layer belonging to the new task.”
Best Practices for Fine-Tuning Visual Classifiers to New Domains (2016, background): “, who both demonstrated substantial variation in the performance of generic features depending on the source dataset and qualitative characteristics of the target dataset [11, 9].”
Transferability and Hardness of Supervised Classification Tasks (2019, background): “For more principled surveys on transfer learning, we refer to others [4, 47, 63, 68].”
Coupled End-to-End Transfer Learning with Generalized Fisher Information (2018, background): “[1] discussed several factors influencing the transferability of features learned by CNN.”
Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN (2017, background): “Fine-tuning is a common strategy in transfer learning with neural networks [18, 24, 2, 15].”