Self-Erasing Network for Integral Object Attention

By Qibin Hou, Peng-Tao Jiang, Yunchao Wei, Ming-Ming Cheng. 2018

301 citations recorded by the source corpus.

Recently, adversarial erasing for weakly-supervised object attention has been deeply studied due to its capability in localizing integral object regions. However, such a strategy raises one key problem that attention regions will gradually expand to non-object regions as training iterations continue, which significantly decreases the quality of the produced attention maps. To tackle such an issue as well as promote the quality of object attention, we introduce a simple yet effective Self-Erasing Network (SeeNet) to prohibit attentions from spreading to unexpected background regions. In particular, SeeNet leverages two self-erasing strategies to encourage networks to use reliable object and background cues for learning to attention. In this way, integral object regions can be effectively highlighted without including much more background regions. To test the quality of the generated attention maps, we employ the mined object regions as heuristic cues for learning semantic segmentation models. Experiments on Pascal VOC well demonstrate the superiority of our SeeNet over other state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Incremental Learning in Semantic Segmentation from Image Labels (2021, background): “Previous works focused on improving the pseudo-labels through multiple refinements steps [2, 3], additional losses [5, 10, 31, 34, 59, 63], or erasing techniques that force the CAM to expand and focus on non-discriminative parts of the image [12, 29, 64].”
Interactive Object Segmentation With Inside-Outside Guidance (2020, methodology): “Particularly, image-level labels [50, 61, 63, 62, 27, 32], points [3, 51], bounding boxes [18, 33] and scribbles [39, 58, 59] have been employed as guidance to supervise the training of semantic segmentation networks.”
Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation (2021, background): “[21], DCSP [8], TPL [27], GAIN [31], DSRG [23], MCOF [48], AffinityNet [2], RDC [52], SeeNet [22], OAA [24], ICD [17], BES [9], Fan et al.”
Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation (2020, methodology): “This property is widely demonstrated by the effectiveness of previous methods [15, 40, 14, 18] because they take the high score regions in CAM as the foreground seeds.”
Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation (2020, unknown): “Adversarial erasing [15, 32] is a popular CAM expansion method, which erases the most discriminative part of CAM, guides the network to learn classification features from other regions and expands activations.”
Puzzle-CAM: Improved Localization Via Matching Partial And Full Features (2021, methodology): “Most previous WSSS methods refine the CAMs generated by the image classifier to approximate the segmentation mask [4, 11, 12, 13, 6].”
Pseudo-mask Matters in Weakly-supervised Semantic Segmentation (2021, methodology): “The methods include adding CRF and dilation [34], designing new losses [17], erasing high response area [13], keeping scale consistency via siamese network and correlation module [33], clustering sub-categories for classification [5].”
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (2020, methodology): “CAM-based methods (Hou et al., 2018; Wei et al., 2017; Ahn & Kwak, 2018) have been successfully adopted to tackle a different weakly supervised semantic segmentation task from us, where they assume only image-level labels are available.”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, methodology): “Besides, because of the popularity of attention methods [1], [2], [3], many WSSS approaches [74], [75], [6], [76], [77], [78], [79], [80], [81] use the object localization cues extracted from the class activation maps.”
EGNet: Edge Guidance Network for Salient Object Detection (2019, background): “It has received widespread attention recently and been widely used in many vision and image processing related areas, such as contentaware image editing [6], object recognition [42], photosynth [4], non-photo-realist rendering [41], weakly supervised semantic segmentation [19] and image retrieval [1”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, methodology): “7 †SeeNet [20] NIPS18 63.”
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation (2022, methodology): “A few methods developed heuristic strategies, such as “Hide & Seek” [31] and Erasing [40], applied either on images [24, 49] or feature maps [16,21] to drive the network to learn novel object patterns.”
Weakly Supervised Segmentation with Maximum Bipartite Graph Matching (2020, methodology): “For example, Seenet [17] incorporates saliency maps into the network training to better inference the background and foreground object regions.”
Single-Stage Semantic Segmentation From Image Labels (2020, methodology): “Similarly, SeeNet [21] implements erasing using two decoder branches aided by saliency [20]; the ﬁrst branch removes the peak CAM-response and feeds into the second.”
Weakly Supervised Scene Parsing with Point-based Distance Metric Learning (2018, background): “And object region mining and localization is experimented in (Wei et al. 2016), (Wei et al. 2017a), (Zhang et al. 2018b) and (Hou et al. 2018).”
Weakly-Supervised Semantic Segmentation via Sub-Category Exploration (2020, methodology): “Recently, the SeeNet approach [17] applies selferasing strategies to encourage networks to use both object and background cues, which prevent the attention from including more background regions.”
Interactive Object Segmentation With Inside-Outside Guidance (2022, methodology): “Particularly, image-level labels [67], [68], [69], [70], [71], [72], points [73], [74], bounding boxes [75], [76], scribbles [77], [78], [79] have been employed as guidance to supervise the training of semantic segmentation networks.”