Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation

By Seunghoon Hong, Hyeonwoo Noh, Bohyung Han. 2015

345 citations recorded by the source corpus.

We propose a novel deep neural network architecture for semi-supervised semantic segmentation using heterogeneous annotations. Contrary to existing approaches posing semantic segmentation as a single task of region-based classification, our algorithm decouples classification and segmentation, and learns a separate network for each task. In this architecture, labels associated with an image are identified by classification network, and binary segmentation is subsequently performed for each identified label in segmentation network. The decoupled architecture enables us to learn classification and segmentation networks separately based on the training data with image-level and pixel-wise class labels, respectively. It facilitates to reduce search space for segmentation effectively by exploiting class-specific activation maps obtained from bridging layers. Our algorithm shows outstanding performance compared to other semi-supervised approaches with much less training images with strong annotations in PASCAL VOC dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully convolutional networks for semantic segmentation (2014, background): “been extended to new tasks and data. Tasks include region proposals [31], contour detection [32], depth regression [33], optical ﬂow [34], and weaklysupervised semantic segmentation [35], [36], [37], [38]. In addition, new works have improved the FCNs presented here to further advance the state-of-t”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, background): “As the per-pixel category labeling is expensive, the supervision signals in FCNs have been relaxed to boxes [6], scribbles [23], or weakly supervised image class labels [19, 20].”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, neutral): “[69], [70], [71], achieving significantly better results than weakly-supervised pre-DCNN systems such as [72].”
Learning Deconvolution Network for Semantic Segmentation (2015, methodology): “Our work is extended to solving the semantic segmentation problem with a small number of full annotations in [12].”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, methodology): “This implicitly favours techniques used for detection as shown by the recent work on a decoupled classification-segmentation network [18] where the classification network can be trained with a large set of weakly labelled data and the independent segmentation network performance is improved.”
Convolutional Neural Fabrics (2016, methodology): “Up-sampling operators are used to increase the resolution of the output, compensating for pooling operators used in earlier layers of the network [24].”
What's the Point: Semantic Segmentation with Point Supervision (2015, background): “Many forms of supervision have been explored, such as eye tracks [25], free-form squiggles [41], noisy web tags [1], or heterogeneous annotations (a large number of image-level labels and a small number of full segmentation masks) [17].”
Learning Multi-domain Convolutional Neural Networks for Visual Tracking (2015, neutral): “Convolutional Neural Networks (CNNs) have recently been applied to various computer vision tasks such as image classification [4, 28, 37], semantic segmentation [19, 31, 33], object detection [12], and many others [34, 40, 41].”
Learning to Adapt Structured Output Space for Semantic Segmentation (2018, background): “As a result, weakly and semi-supervised approaches [5, 14, 17, 28, 29] are proposed in recent years to reduce the heavy labeling cost of collecting segmentation ground truths.”
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation (2016, methodology): “…space, we introduce a gener-alizable constrained multiple instance loss function, which expands on weak label learning [26, 25, 27, 24, 14], but can be applied to the target domain without any extra annotations and explicitly transfers category layout information from a labeled source…”
Large-Scale Image Retrieval with Attentive Deep Local Features (2016, background): “Many visual recognition problems employ visual attention based on deep neural networks, which include object detection [45], semantic segmentation [14], image captioning [38], visual question answering [39], etc.”
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes (2017, background): “Further works have focused on relaxing the level of supervision in FCNs by considering bounding boxes [7], scribbles [27], or image class labels [20, 21] as sources of weak supervision.”
Identity-Guided Human Semantic Parsing for Person Re-Identification (2020, background): “semantic segmentation [16, 47, 19, 12, 11, 33, 44, 62], which aim to locate objects like person, horse or dog at pixel-level with image-level supervision.”
A Survey on Deep Semi-Supervised Learning (2021, background): “Semantic segmentation PASCAL VOC [65], PASCAL context [80], MS COCO [66], Cityscapes [81], CamVid [82], SiftFlow [83], [84], StanfordBG [85] [50], [86], [87], [88], [89], [90], [91], [92], [93], [94]”
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes (2017, methodology): “As with most recent image segmentation networks [10], [8], [26], [18], [11], MFNet adopted the Encoder-Decoder structure.”
Simple Does It: Weakly Supervised Instance and Semantic Segmentation (2016, background): “[8, 27, 15] also consider the case where a fraction of images are fully supervised.”
Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images (2017, background): “Hence, a natural question is: How could we utilize unannotated images to benefit and improve segmentation? Some recent attempts [5,7] were made to utilize weakly annotated images in natural scene image segmentation.”
Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes (2019, background): “, with only image labels or only bounding boxes) [25, 31, 37, 39, 13, 6, 14, 32] or completely unlabeled [24, 29, 15, 31, 19] data after a first stage of supervised training.”