Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning

By Yin Cui, Yang Song, Chen Sun, Andrew G. Howard, Serge J. Belongie. 2018

538 citations recorded by the source corpus.

Transferring the knowledge learned from large scale datasets (e.g., ImageNet) via fine-tuning offers an effective solution for domain-specific fine-grained visual categorization (FGVC) tasks (e.g., recognizing bird species or car make & model). In such scenarios, data annotation often calls for specialized domain knowledge and thus is difficult to scale. In this work, we first tackle a problem in large scale FGVC. Our method won first place in iNaturalist 2017 large scale species classification challenge. Central to the success of our approach is a training scheme that uses higher image resolution and deals with the long-tailed distribution of training data. Next, we study transfer learning via fine-tuning from large scale datasets to small scale, domain-specific FGVC datasets. We propose a measure to estimate domain similarity via Earth Mover's Distance and demonstrate that transfer learning benefits from pre-training on a source domain that is similar to the target domain by this measure. Our proposed transfer learning outperforms ImageNet pre-training and obtains state-of-the-art results on multiple commonly used FGVC datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Grafit: Learning fine-grained image representations with coarse labels (2020, background): “It is preferable to pre-train on a domain similar to the target [16], e.”
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Recent work [9, 3, 33, 71, 37] has pointed out the effectiveness of two-stage training strategies by separating the feature learning and the re-balancing stage, as end-to-end training with re-balancing strategies could be detrimental to feature learning.”
Concept Bottleneck Models (2020, methodology): “We follow the preprocessing practices described in Cui et al. (2018).”
Large-Scale Long-Tailed Recognition in an Open World (2019, background): “Arising from long-tail distributions of natural data, it has been extensively studied [41, 61, 3, 30, 62, 34, 29, 49, 6].”
Decoupling Representation and Classifier for Long-Tailed Recognition (2019, methodology): “Recent approaches (Cui et al., 2018; Cao et al., 2019) utilized mixed ways of sampling, i.”
Domain Adaptive Transfer Learning with Specialist Models (2018, methodology): “[9] and Ge & Yu [11] recently proposed methods for improving transfer learning by selecting relevant data from the source pre-training dataset.”
Does learning require memorization? a short tale about a long tail (2019, background): “Specifically, these problems often have more than 10, 000 fine-grained labels and the number of examples per class is longtailed [3, 4, 21, 30, 49, 51].”
Class-Balanced Loss Based on Effective Number of Samples (2019, background): “It is noteworthy to mention that previous work has also explored other ways of dealing with data imbalance, including transferring the knowledge learned from major classes to minor classes [3, 31, 43, 6, 44] and designing a better training objective via metric learning [17, 47, 45].”
Locality and compositionality in zero-shot learning (2019, background): “First, relative success in transfer learning has been shown to be highly dependent on the precise instantiation of the pretrained backbone encoder (Xian et al., 2018) or the pre-training dataset (Cui et al., 2018).”
Scaling Vision Transformers to 22 Billion Parameters (2023, methodology): “We further test linear separability on the ﬁne-grained clas-siﬁcation dataset, iNaturalist 2017 (Cui et al., 2018 (Zhai et al., 2022a) on the English subset of the WebLI dataset (Chen et al., 2022) for 1M steps with a 32K batch size.”
Imbalance Problems in Object Detection: A Review (2019, methodology): “Another study [142] adopted a different strategy during transfer learning: Firstly, the network is trained with the entire imbalanced dataset, and then the resulting network is fine-tuned by using a balanced subset of the dataset.”
Learning What and Where to Transfer (2019, background): “Cui et al. (2018) suggest to sample from the source dataset depending on a target task for pre-training, but it is only possible when the source dataset is available.”
When Does Self-supervision Improve Few-shot Learning? (2019, methodology): “[12] used Earth Mover’s distance to measure the domain similarity and select the source domain for pretraining.”
On-Device Training Under 256KB Memory (2022, background): “There are several ways to reduce the transfer learning cost compared to fine-tuning the full model [39, 22, 38].”
TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning (2020, methodology): “, BN, GN) and the last linear layer [42] (referred to as FT-Norm+Last) ; iii) fine-tuning the full network [43, 44] (referred to as FT-Full).”
ELF: An Early-Exiting Framework for Long-Tailed Classification (2020, background): “Transfer and meta learning based approaches aims to transfer knowledge from the majority classes to rare ones by transfer learning, multi task learning or learning to learn [12, 27, 28].”
BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition (2019, background): “In addition, recently, some two-stage fine-tuning strategies [3, 6, 22] were developed to modify re-balancing for effectively handling long-tailed problems.”