Multi-evidence Filtering and Fusion for Multi-label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning

By Weifeng Ge, Sibei Yang, Yizhou Yu. 2018

196 citations recorded by the source corpus.

Supervised object detection and semantic segmentation require object or even pixel level annotations. When there exist image level labels only, it is challenging for weakly supervised algorithms to achieve accurate predictions. The accuracy achieved by top weakly supervised algorithms is still significantly lower than their fully supervised counterparts. In this paper, we propose a novel weakly supervised curriculum learning pipeline for multi-label object recognition, detection and semantic segmentation. In this pipeline, we first obtain intermediate object localization and pixel labeling results for the training images, and then use such results to train task-specific deep networks in a fully supervised manner. The entire process consists of four stages, including object localization in the training images, filtering and fusing object instances, pixel labeling for the training images, and task-specific network training. To obtain clean object instances in the training images, we propose a novel algorithm for filtering, fusing and classifying object instances collected from multiple solution mechanisms. In this algorithm, we incorporate both metric learning and density-based clustering to filter detected object instances. Experiments show that our weakly supervised pipeline achieves state-of-the-art results in multi-label image classification as well as weakly supervised object detection and very competitive results in weakly supervised semantic segmentation on MS-COCO, PASCAL VOC 2007 and PASCAL VOC 2012.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Elucidating image-to-set prediction: An analysis of models, losses and datasets (2019, methodology): “2018 [14] ResNet-101 3 82081 n/a 40504 76.”
Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, methodology): “Following the convention of multi-label classification [84,97,30], we report the average overall F1 (O-F1), per-class F1 (C-F1) as well as the mAP.”
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection (2018, methodology): “We can expect to obtain better detection performance through replacing our conference version method in [57] by our newly proposed method here, which we would”
ELASTIC: Improving CNNs With Dynamic Scaling Policies (2018, methodology): “Following the literature of multilabel classification[41, 7, 39, 17], results are evaluated using macro/micro evaluations.”
Dissimilarity Coefficient Based Weakly Supervised Object Detection (2018, background): “Typically, the predicted instances from a trained weakly supervised detector are treated as a pseudo-strong label to train a strongly supervised network [12, 22, 33, 34, 40, 41, 42].”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, unknown): “*MEFF [13] CVPR18 - 55.”
Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up (2019, background): “in [12] progressively mine the object locations and pixel labels with the filtering and fusion of multiple evidences.”
C-MIDN: Coupled Multiple Instance Detection Network With Segmentation Guidance for Weakly Supervised Object Detection (2019, methodology): “Some methods propose to leverage Weakly Supervised Semantic Segmentation to improve WSOD [12, 9, 40].”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, background): “In order to handle the data deficiency problem in the image segmentation task, researchers have explored various types of weak supervision, such as bounding boxes [8, 39], scribbles [32, 33], and image labels[2, 3, 12, 22, 27, 42, 44, 46–48, 53, 66, 67, 67].”
General Multi-label Image Classification with Transformers (2020, unknown): “For fair comparisons to previous works [16, 58], we also consider the setting where we evaluate the Top-3 predicted labels following.”
Detecting 11K Classes: Large Scale Object Detection Without Fine-Grained Bounding Boxes (2019, methodology): “weakly-supervised methods are still 30 points below fullysupervised methods [10, 39, 33] in mAP, while our semisupervised method demonstrates that we can achieve comparable performance to fully-supervised methods.”