The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition

By Jonathan Krause, Benjamin Sapp, Andrew Howard, Howard Zhou, Alexander Toshev, Tom Duerig, James Philbin, Fei-Fei Li. 2015

374 citations recorded by the source corpus.

Current approaches for fine-grained recognition do the following: First, recruit experts to annotate a dataset of images, optionally also collecting more structured data in the form of part annotations and bounding boxes. Second, train a model utilizing this data. Toward the goal of solving fine-grained recognition, we introduce an alternative approach, leveraging free, noisy data from the web and simple, generic methods of recognition. This approach has benefits in both performance and scalability. We demonstrate its efficacy on four fine-grained datasets, greatly exceeding existing state of the art without the manual collection of even a single label, and furthermore show first results at scaling to more than 10,000 fine-grained categories. Quantitatively, we achieve top-1 accuracies of \(92.3\,\%\) on CUB-200-2011, \(85.4\,\%\) on Birdsnap, \(93.4\,\%\) on FGVC-Aircraft, and \(80.8\,\%\) on Stanford Dogs without using their annotated training sets. We compare our approach to an active learning approach for expanding fine-grained datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “In order to overcome the bottleneck, there have been recent efforts on visual representation learning using web-supervision [2, 5, 6, 9, 21, 23, 24, 27] or unsupervised [10, 11, 31, 32, 34, 42, 43] paradigms.”
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “Although, these datasets cover a significant number of images (e.g., about 80k in MSCOCO and 30K in Flickr30K), creating a larger dataset with image-sentence pairs is extremely difficult and labor-intensive [32].”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017, background): “Accuracy Mult-Adds Parameters Inception V3 [18] 84% 5000 23.”
Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild (2019, background): “However, in the real world, we live with a continuous and large stream of unlabeled data easily obtainable on the ﬂy or transiently, e.g., by data mining on social media [26] and web data [17].”
Guided Open Vocabulary Image Captioning with Constrained Beam Search (2016, background): “Even if these annotations do not exist, they can be generated using (potentially task speciﬁc) image taggers (Chen et al., 2013; Zhang et al., 2016) or object detectors (Ren et al., 2015; Krause et al., 2016), which are easier to scale to new concepts.”
Domain Adaptive Transfer Learning with Specialist Models (2018, methodology): “[17] obtained additional noisy training examples by searching the web with the class labels.”
The Devil is in the Tails: Fine-grained Classification in the Wild (2017, background): “[26] investigated the collection and use of a large, noisy dataset for the task of ﬁne-grained classiﬁcation and found that off the shelf CNN models can readily take advantage of these datasets to increase accuracy and reach state-of-the-art performance.”
On the genealogy of machine learning datasets: A critical history of ImageNet (2021, background): “…Recognition, ” co-authored by Fei-Fei Li with researchers at Stanford University and Google, discusses how images gathered from the web, even without carefully cultivated annotations and labels, can still produce very good results for the image classi ﬁ cation task (Krause et al., 2016).”
The iNaturalist Challenge 2017 Dataset (2017, methodology): “Unlike web scraped datasets [16, 15, 43], the annotations in iNat2017 have all been collected from the consensus of informed enthusiasts.”
Learning from Noisy Labels with Distillation (2017, methodology): “discovered that, in the case of fine grained classification, the label noise does not hurt the performance, because the noisy examples are not shown in the evaluation [10].”
Bootstrapping the Performance of Webly Supervised Semantic Segmentation (2018, methodology): “Our method is closely related to webly supervised learning [4, 14, 31, 5], which is focused on extracting useful knowledge or features from noisy web data.”
CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture (2019, neutral): “Unlike web scraped datasets [21], the annotations in iNat2017 represent the consensus of informed enthusiasts.”
Fully Convolutional Attention Localization Networks: Efficient Attention Localization for Fine-Grained Recognition (2016, methodology): “Although the result of the proposed is not the best one on this dataset, it is simpler to implement than [23] and uses less time for training and testing than [18].”
Recognition From Web Data: A Progressive Filtering Approach (2018, methodology): “In this area, impressive improvement has been observed [6], [11], [12], [14]–[17].”