FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference

By Jungbeom Lee, Eunji Kim, Sungmin Lee, Jangho Lee, Sungroh Yoon. 2019

451 citations recorded by the source corpus.

The main obstacle to weakly supervised semantic image segmentation is the difficulty of obtaining pixel-level information from coarse image-level annotations. Most methods based on image-level annotations use localization maps obtained from the classifier, but these only focus on the small discriminative parts of objects and do not capture precise boundaries. FickleNet explores diverse combinations of locations on feature maps created by generic deep neural networks. It selects hidden units randomly and then uses them to obtain activation scores for image classification. FickleNet implicitly learns the coherence of each location in the feature maps, resulting in a localization map which identifies both discriminative and other parts of objects. The ensemble effects are obtained from a single network by selecting random hidden unit pairs, which means that a variety of localization maps are generated from a single image. Our approach does not require any additional training steps and only adds a simple layer to a standard convolutional neural network; nevertheless it outperforms recent comparable techniques on the Pascal VOC 2012 benchmark in both weakly and semi-supervised settings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Incremental Learning in Semantic Segmentation from Image Labels (2021, methodology): “To address this issue, Weakly Supervised Semantic Segmentation (WSSS) methods aim to obtain effective segmentation models using cheaper supervisions such as bounding boxes [18, 32, 49], scribbles [39, 60], points [6, 53], and image-level labels [31, 34, 35, 59].”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, background): “Another line of related research is of weaklysupervised semantic segmentation [1,9,26,31,36,40,65,68, 80], which aims to learn semantic segmentation with only image-level object category supervision.”
Puzzle-CAM: Improved Localization Via Matching Partial And Full Features (2021, methodology): “Most previous WSSS methods refine the CAMs generated by the image classifier to approximate the segmentation mask [4, 11, 12, 13, 6].”
Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation (2020, methodology): “Previous approaches try to alleviate this problem by adopting dilated convolutions [40], iterative erasing strategy [38], randomly dropping connections [22], and online accumulation strategy [18], etc.”
Single-Stage Semantic Segmentation From Image Labels (2020, background): “[30] stochastically selected hidden units and supplied the improved initial seeds to DSRG [24].”
An Overview of Deep Semi-Supervised Learning (2020, background): “For example, in weakly-supervised semantic segmentation, pixel-level labels, which are harder and more expensive to acquire, are substituted for inexact annotations, e . g ., image labels [159, 184, 161, 97, 94], points [9], scribbles [100] and bounding boxes [144, 31].”
Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (2021, background): “As a more generalized approach, FickleNet (Lee et al. 2019) aggregate diverse localization maps produced by stochastic feature selection.”
Semi-supervised Medical Image Segmentation through Dual-task Consistency (2020, methodology): “Weakly supervised learning methods learn from bounding boxes, scribbles or image-level tags for image segmentation rather than using pixel-wise annotation, which reduces the burden of annotation (Dai, He, and Sun 2015; Lin et al. 2016; Lee et al. 2019).”
Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation (2021, methodology): “Following the WSSS convention [13, 21, 27, 28, 41, 42], we then train a segmentation model, where the generated pseudo-masks in the first stage are used as supervision.”
Simultaneously Localize, Segment and Rank the Camouflaged Objects (2021, background): “[63] introduced the class activation map (CAM) to estimate the discriminative region of each class, which is the basis of many weakly supervised methods [1, 51, 17, 50, 24, 43, 47].”
Causal Intervention for Weakly-Supervised Semantic Segmentation (2020, methodology): “To this end, existing methods mainly focus on generating better seed areas [25, 57, 59, 58] and expanding these seed areas [1, 2, 20, 23, 55].”
A Survey on Deep Semi-Supervised Learning (2021, background): “Semantic segmentation PASCAL VOC [65], PASCAL context [80], MS COCO [66], Cityscapes [81], CamVid [82], SiftFlow [83], [84], StanfordBG [85] [50], [86], [87], [88], [89], [90], [91], [92], [93], [94]”
Guided Collaborative Training for Pixel-wise Semi-Supervised Learning (2020, background): “[26] study weak-supervised learning in the SSL context.”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, methodology): “yers are replaced with three convolutional layers with 512 channels and kernel size 33, as done in [23,83]. For the semantic segmentation network, for fair comparison with current top-leading methods [2,23,31,55], we adopt the ResNet-101 [17] version Deeplab-LargeFOV architecture. Training Phases of”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, unknown): “(Li et al., 2018) VGG16 60.50 MDC (Wei et al., 2018) VGG16 65.70 DSRG (Huang et al., 2018) VGG16 64.30 GANSeg (Souly et al., 2017) VGG16 65.80 FickleNet (Lee et al., 2019) ResNet-101 65.80 CCT (Ouali et al., 2020) ResNet-50 73.20 PseudoSeg (Ours) ResNet-50 73.80 Similar to semi-supervised learning u”
Identity-Guided Human Semantic Parsing for Person Re-Identification (2020, methodology): “ision To the best of our knowledge, there is no previous work to learning human semantic parsing with image-level supervision but only weakly-supervised methods for 4 Zhu et al. semantic segmentation [16,47,19,12,11,33,44,62], which aim to locate objects like person, horse or dog at pixel-level with”
Weakly-Supervised Semantic Segmentation via Sub-Category Exploration (2020, methodology): “Compared to the recent FickleNet [24] method that also focuses on improving the initial response map, the proposed algorithm performs favorably for the segmentation task in terms of the mean IoU.”