Puzzle-CAM: Improved Localization Via Matching Partial And Full Features

By Sanghyun Jo, In-Jae Yu. 2021

131 citations recorded by the source corpus.

Weakly-supervised semantic segmentation (WSSS) is introduced to narrow the gap for semantic segmentation performance from pixel-level supervision to image-level supervision. Most advanced approaches are based on class activation maps (CAMs) to generate pseudo-labels to train the segmentation network. The main limitation of WSSS is that the process of generating pseudo-labels from CAMs that use an image classifier is mainly focused on the most discriminative parts of the objects. To address this issue, we propose Puzzle-CAM, a process that minimizes differences between the features from separate patches and the whole image. Our method consists of a puzzle module and two regularization terms to discover the most integrated region in an object. Puzzle-CAM can activate the overall region of an object using image-level supervision without requiring extra parameters. In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 dataset. Code associated with our experiments is available at https://github.com/OFRIN/PuzzleCAM.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yin Cui
2. Tsung-Yi Lin
3. Golnaz Ghiasi
4. Xiuye Gu
5. X. Wang
6. Jan Kautz
7. Jiarui Xu
8. Sifei Liu

In Computer Vision and Pattern Recognition

1. Ming-Hsuan Yang
2. Luc Van Gool
3. Dacheng Tao
4. Trevor Darrell
5. Chen Change Loy
6. Alan L. Yuille
7. Xiaogang Wang
8. Jitendra Malik

Citations in the corpus

• GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, background): “Another line of related research is of weaklysupervised semantic segmentation [1,9,26,31,36,40,65,68, 80], which aims to learn semantic segmentation with only image-level object category supervision.”
• Open-Vocabulary Image Segmentation (2021, background): “Weaklysupervised semantic segmentation can train with imagelevel labels [23,29,34,45,49], of which refining CAMs [53] is a popular techniques.”
• Scaling Open-Vocabulary Image Segmentation with Image-Level Labels (2021, background): “Semantic segmentation with less supervisions: Weakly-supervised semantic segmentation trains with image-level labels [36,47,24,31,52], of which refining CAMs [60] is a popular techniques.”