Best Practices for Fine-Tuning Visual Classifiers to New Domains

By Brian Chu, Vashisht Madhavan, Oscar Beijbom, Judy Hoffman, Trevor Darrell. 2016

161 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Do Better ImageNet Models Transfer Better? (2018, background): “A substantial body of existing research indicates that, in image tasks, fine-tuning typically achieves higher accuracy than classification based on fixed features, especially for larger datasets or datasets with a larger domain mismatch from the training set [1, 6, 24, 74, 2, 43, 33, 9, 51].”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “While there is a large body of work on transfer learning, much of it assumes a fixed capacity model [32, 3, 6, 58, 15].”
Meta-learning with differentiable closed-form solvers (2018, background): “…also one of the main traits that differentiate our approach from basic transfer learning approaches in which a neural network is ﬁrst pre-trained on one dataset/task and then adapted to a different dataset/task by simply adapting the ﬁnal layer(s) (e.g. Yosinski et al. (2014); Chu et al. (2016)).”
IL2M: Class Incremental Learning With Dual Memory (2019, methodology): “An end-to-end IL (EtEIL) algorithm is introduced in [6] which also exploits a combined loss.”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, background): “The second and most used solution is to fine-tune the deep network model on the new type of data and for the new task [132, 133, 134, 135].”
Self-Similarity Grouping: A Simple Unsupervised Cross Domain Adaptation Approach for Person Re-Identification (2018, background): “Some works in this community try to address this problem by reducing the discrepancy between source domain and target domain [6, 34, 42].”
Small Sample Learning in Big Data Era (2018, methodology): “In SSL, fine-tuning has been successfully applied to visual classifiers in new domains (Chu et al., 2016), object detection with long-tail distribution (Ouyang et al.”
Progressive Transfer Learning and Adversarial Domain Adaptation for Cross-Domain Skin Disease Classification (2020, unknown): “In image representation and label space, it is common to observe a large data distribution shift [37], [38] from large-scale source domain datasets such as ImageNet [24] to medium/small size target domain datasets such as Pascal VOC [39] and CIFAR100 [40].”
IDDA: A Large-Scale Multi-Domain Dataset for Autonomous Driving (2020, methodology): “1 in [26]) we extract, using ResNet-101 [17] pretrained on ImageNet, the features of the first 500 samples of each domain and we reduce the dimensionality (using PCA) taking the first 50 principal components.”
Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences (2017, background): “Research in computer vision suggests that features from deep CNNs performs well as generic features when classifying images in new domains [35].”
Intelligent Autonomous Things on the Battlefield (2019, methodology): “Whether by updating only a small portion of the model (Chu et.al.”
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning (2022, unknown): “Existing CIL approaches (He & Garcia, 2009; Buda et al., 2018; Rebuﬃ et al., 2017; Belouadah & Popescu, 2019; 2020; Chu et al., 2016; Kemker & Kanan, 2018; Parisi et al., 2019) We present our novel remedy of utilizing anchor-based unlabeled data query, to balance between preserving the historical…”