Webly Supervised Learning of Convolutional Networks

By Xinlei Chen, Abhinav Kumar Gupta. 2015

400 citations recorded by the source corpus.

We present an approach to utilize large amounts of web data for learning CNNs. Specifically inspired by curriculum learning, we present a two-step approach for CNN training. First, we use easy images to train an initial visual representation. We then use this initial CNN and adapt it to harder, more realistic images by leveraging the structure of data and categories. We demonstrate that our two-stage CNN outperforms a fine-tuned CNN trained on ImageNet on Pascal VOC 2012. We also demonstrate the strength of webly supervised learning by localizing objects in web images and training a R-CNN style [19] detector. It achieves the best performance on VOC 2007 where no VOC training data is used. Finally, we show our approach is quite robust to noise and performs comparably even when we use image search results from March 2013 (pre-CNN image search era).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “These image-query pairs are also often used to improve performance on standard datasets as additional training data (Chen & Gupta, 2015).”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “In order to overcome the bottleneck, there have been recent efforts on visual representation learning using web-supervision [2, 5, 6, 9, 21, 23, 24, 27] or unsupervised [10, 11, 31, 32, 34, 42, 43] paradigms.”
Learning Visual Representations with Caption Annotations (2020, background): “h the signal-to-noise ratio of samples crawled from the web may arguably be lower than carefullyconstructed datasets, signicant progress has been made leveraging this type of data to pretrain models [10,28]. Among those, to learn visual representations, [32] extracts the most common hashtags and wor”
Learning Video Representations from Textual Web Supervision (2020, methodology): “The most commonly-used approach is to use image search results, and to label each image with the query that was used to ﬁnd it [65, 15, 56, 6, 11, 13, 21, 10, 18, 35].”
Learning Visual Features from Large Weakly Supervised Data (2015, background): “In particular, Chen and Gupta [6] propose a curriculum-learning approach that trains convolutional networks on “easy” examples retrieved from Google Images, and then ﬁnetunes the models on weakly labeled image-hashtag pairs.”
RegionCLIP: Region-based Language-Image Pretraining (2021, background): “The most relevant work is learning visual representations from natural language, such as image tags [3, 8, 12, 25, 28] and text descriptions [11,23,43,53,62].”
Grounding of Textual Phrases in Images by Reconstruction (2015, background): “phrases and allows for localization of novel phrases. There are also works that propose to train detectors for a wide range of concepts using image-level annotated data from web image search [17] and [26]. These approaches are complementary to ours in the sense of obtaining large scale concept detec”
AutoML: A Survey of the State-of-the-Art (2019, background): “A common solution is to fine-tune these web data [68], [69].”
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, methodology): “Webly supervised learning [168] is a technique for training a convolutional neural network on clean and noisy images on the Web.”
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, methodology): “4 is inspired by methods that propose to learn from web data [13, 8, 9, 60, 40, 29, 16] or reason about label noise [34, 57, 52, 47, 38].”
RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, background): “Webly-Supervised Learning is an emerging paradigm in which large amounts of web data is exploited for learning CNNs [8, 10, 32].”
Deep Anomaly Detection with Outlier Exposure (2018, background): “Chen & Gupta (2015); Mahajan et al. (2018) show that representations learned from images scraped from the nigh unlimited source of search engines and photo-sharing websites improve object detection performance.”
DeeSIL: Deep-Shallow Incremental Learning (2018, methodology): “nging dataset which is obtained from weakly annotated Flickr group data and is visually more distant from the 4 Eden Belouadah, Adrian Popescu test set. Within each group, a semi-supervised reranking [12] is initially performed to remove a part of noisy images. A greedy algorithm [13] which operates”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “Dataset sources: Many prior approaches use Internet as a natural source of content [6, 12, 13, 18, 24, 25, 34, 37, 44, 46, 51, 58, 67] and the associated search queries, hashtags, or user-comments as labels to construct datasets.”
Learning from Noisy Large-Scale Datasets with Minimal Supervision (2017, background): “ing [17], self-supervised learning [9, 24, 25, 31] and learning from noisy annotations [6, 23].”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “268, 269, 270, 95] or multi-view embedding [93, 110, 99, 111, 112, 113]. Hence, HDA/HTL can strongly beneﬁt from such cross-modal data representations. In the same spirit, webly supervised approaches [271, 272, 273, 274, 158, 159, 275] are also related to DA and HDA as is these approaches rely on co”
Learning to Reweight Examples for Robust Deep Learning (2018, background): “Coarse labels can come from inexpensive crowdsourcing services or weakly supervised data (Cordts et al., 2016; Russakovsky et al., 2015; Chen & Gupta, 2015).”
Hard Mixtures of Experts for Large Scale Weakly Supervised Vision (2017, positive): “Recent works [4, 2, 8] have shown that it is possible to build vision models with weakly supervised instead of hand-annotated data, and open the possibility of using truly gigantic datasets.”
Learning from Noisy Labels with Distillation (2017, background): “Chen et al. proposed a two-step approach to learn ConvNet by image search on the Internet [3].”