Enriching Visual Knowledge Bases via Object Discovery and Segmentation

By Xinlei Chen, Abhinav Shrivastava, Abhinav Kumar Gupta. 2014

126 citations recorded by the source corpus.

There have been some recent efforts to build visual knowledge bases from Internet images. But most of these approaches have focused on bounding box representation of objects. In this paper, we propose to enrich these knowledge bases by automatically discovering objects and their segmentations from noisy Internet images. Specifically, our approach combines the power of generative modeling for segmentation with the effectiveness of discriminative models for detection. The key idea behind our approach is to learn and exploit top-down segmentation priors based on visual subcategories. The strong priors learned from these visual subcategories are then combined with discriminatively trained detectors and bottom up cues to produce clean object segmentations. Our experimental results indicate state-of-the-art performance on the difficult dataset introduced by [29] Rubinstein et al. We have integrated our algorithm in NEIL for enriching its knowledge base [5]. As of 14th April 2014, NEIL has automatically generated approximately 500K segmentations using web data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Webly Supervised Learning of Convolutional Networks (2015, methodology): “Clustering into Subcategories: We then use a publiclyavailable variant of agglomerative clustering [10] where the nearest neighbor sets are merged iteratively from bottom up to form the final subcategories based on E-LDA similarity scores and density estimation.”
iCaRL: Incremental Classifier and Representation Learning (2016, background): “…standard classification loss, which encourages improvements of the feature representation that allow classifying the newly observed classes well, it also contains the distillation loss, which ensures that the discriminative information learned previously is not lost during the new learning step.”
Training Deep Neural Networks on Noisy Labels with Bootstrapping (2014, background): “Never ending language learning (NELL) (Carlson et al., 2010) and never ending image learning (NEIL) (Chen et al., 2013; 2014) are lifelong-learning systems for language and image understanding, respectively.”
Explicit Knowledge-based Reasoning for Visual Question Answering (2015, background): “, YAGO [22, 28], OpenIE [3, 14, 15], NELL [9], NEIL [10, 11]).”
FVQA: Fact-Based Visual Question Answering (2016, methodology): “ by manual annotation (e.g., DBpedia [23], Freebase [25] and Wikidata [29]), or by automatic extraction from unstructured/semi-structured data (e.g., YAGO [28, 33], OpenIE [24,34,35], NELL [26], NEIL [27,36], WebChild [37,38], ConceptNet [39]). The KB we use here is 4 P. Wang, Q. Wu, C. Shen, A. van”
Computational Baby Learning (2014, methodology): “For the first category, existing image-based approaches [21] [2] [6] [3] iteratively used image search and detection results to cover more variations.”
The Devil of Face Recognition is in the Noise (2018, background): “Search engines usually operate in a high-precision regime [2].”
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences (2015, background): “But going from coarse to fine-grained pixel-wise correspondence is still very much an open problem, the solution to which could benefit many vision and graphics tasks, including image edit propagation [14, 46], co-segmentation [34, 7], structurefrom-motion [36], 3D object reconstruction [4, 39] unsu”
Contextual Priming and Feedback for Faster R-CNN (2016, background): “In fact, there has been a lot of interest in using segmentation in tandem with detection [30, 31, 37, 38]; e.”
Sense discovery via co-clustering on images and text (2015, methodology): “For the evaluation of visual senses, since it is hard to obtain ground truth labels, we computed the purity (how many instances belong to the same semantic sense) for all the visual senses obtained by [10] and our approach.”
Detection of Co-salient Objects by Looking Deep and Wide (2016, methodology): “Most previous approaches for co-saliency detection explore the joint information provided by the image pair (Li and Ngan 2011; Chen et al. 2010; Jacobs et al. 2010; Tan et al. 2013). Jacobs et al. (2010) defined co-saliency as the importance, or saliency, of image pixels in the context of multiple i”
Learn To Pay Attention (2018, unknown): “G (Arbel´aez et al., 2014) 32.02 54.21 37.85 — Joint segmentation-based — Joulin et al.(Joulin et al., 2010) 15.36 37.15 30.16 Object-discovery (Rubinstein et al., 2013) 55.81 64.42 51.65 Chen et al.(Chen et al., 2014) 54.62 69.20 44.46 Jain et al.(Dutt Jain & Grauman, 2016) 58.65 66.47 53.57 Fi”
Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection (2016, background): “Visual subcategories are constructed by clustering in the appearance space of object [6, 22, 5, 7].”
One-Shot Learning for Semantic Segmentation (2017, background): “Many co-segmentation algorithms [8, 17, 34] assume object visual appearances in a batch are similar and either rely on hand-tuned low-level features or high-level CNN features trained for different tasks or objects [34].”
Learning Everything about Anything: Webly-Supervised Visual Concept Learning (2014, unknown): “Hence it is possible to cosegment the instances, and learn a foreground segmentation model for each component using cosegmentation [8, 21, 41].”
Understanding Blooming Human Groups in Social Networks (2015, background): “There have been some recent efforts on building visual knowledge bases from Internet images and improving the initial models by using very few labeled data [2], [3].”
Webly Supervised Semantic Segmentation (2017, methodology): “Webly Supervised Computer Vision The idea of utilizing web images for supervising computer vision algorithms has been explored in several tasks, such as object classification [28], object detection [29], object parts localization [30] and object segmentation [17] [31] [32] [33].”