Fine-Grained Recognition in the Wild: A Multi-task Domain Adaptation Approach

By Timnit Gebru, Judy Hoffman, Li Fei-Fei. 2017

164 citations recorded by the source corpus.

While fine-grained object recognition is an important problem in computer vision, current models are unlikely to accurately classify objects in the wild. These fully supervised models need additional annotated images to classify objects in every new scenario, a task that is infeasible. However, sources such as e-commerce websites and field guides provide annotated images for many classes. In this work, we study fine-grained domain adaptation as a step towards overcoming the dataset shift between easily acquired annotated images and the real world. Adaptation has not been studied in the fine-grained setting where annotations such as attributes could be used to increase performance. Our work uses an attribute based multi-task adaptation loss to increase accuracy from a baseline of 4:1% to 19:1% in the semi-supervised adaptation case. Prior domain adaptation works have been benchmarked on small datasets such as [46] with a total of 795 images for some domains, or simplistic datasets such as [41] consisting of digits. We perform experiments on a subset of a new challenging fine-grained dataset consisting of 1; 095; 021 images of 2; 657 car categories drawn from e-commerce websites and Google Street View.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, methodology): “At a high level these methods comprise two subtypes: fine-tuning over target domain (Long et al., 2016; Hoffman et al., 2017; Motiian et al., 2017a; Gebru et al., 2017; Volpi et al., 2018) and coercing domain invariance via adversarial learning (or further extensions) (Ganin et al.”
Recognition in Terra Incognita (2018, methodology): “[43] demonstrated results of a similar method for fine-grained classification,”
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning (2018, background): “To deal with the lack of training data in FGVC, additional web images can be collected to augment the original dataset [10, 31, 62, 18].”
Domain Adaptive Faster R-CNN for Object Detection in the Wild (2018, background): “Recently there are some works concerning tasks such as semantic segmentation [4, 27, 61], fine-grained recognition [16] etc.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “research is needed in areas such as domain adaptation [207], [208], [209], long-tailed distributions [210], [211], open world settings [212], scale variations [2], fine-grained video understanding [213], [214], knowledge transfer, and resource constrained embedded deployment, to name a few.”
Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift (2018, background): “While early UDA studies mainly focus on shallow models [37] [15], with the dramatic up-surge of convolutional neural networks (CNNs), deep UDA has emerged as a thriving solution and has achieved many stunning results [20] [4] [12].”
Unsupervised Domain Adaptation with Similarity Learning (2017, background): “The domain-confusion is achieved in two different ways: (i) discrepancy loss, that reduces the shift between the two domains [51, 55, 36, 38, 57, 16] or (ii) adversarial loss, which encourages a common feature space with respect to a discriminator [53, 14, 15, 1, 54].”
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks (2019, background): “8.6% conﬁdence. As the chins of 0bull mastiff 0and 0pug are pretty indiscriminate, therefore the generated counterfactual explanation is consistent with humans. Recently, as ﬁne-grained classiﬁcation [13] receives more attention which classiﬁes similar classes such as 0bull mastiff 0and pug , we bel”
Deep Learning in the Wild (2018, background): “ted sheet music. These test pictures can have a wide variety of impairments, such as bad printer quality, torn or stained paper etc. While some work has been published on the topic of domain transfer [19], the results are non-satisfactory. The core idea to address this problem here is transfer learn”
Extracting Relationships by Multi-Domain Matching (2018, background): “With the increasing use of neural networks, weight sharing and transfer has emerged as an effective strategy for domain adaptation [15].”
Deep Visual Domain Adaptation: A Survey (2018, methodology): “[22] drew inspiration from these works and leveraged attributes to improve performance in the DA of fine-grained recognition.”
FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery (2018, methodology): “Some methods require additional part [58, 6, 55], attribute [14], or text [39, 19] annotations.”
A Unified Framework for Multimodal Domain Adaptation (2018, background): “There are mainly three kinds of constraints for fine-tuning the deep model, class criterion [17, 38],statistic criterion [37, 39] and architecture criterion [25, 35].”
Deep learning-enabled medical computer vision (2021, methodology): “Within medicine, ﬁ ne-grained versions of these methods 15 have successfully been applied to the classi ﬁ cation and detection of many diseases (Fig.”