Triply Supervised Decoder Networks for Joint Detection and Segmentation

By Jiale Cao, Yanwei Pang, Xuelong Li. 2018

68 citations recorded by the source corpus.

Joint object detection and semantic segmentation is essential in many fields such as self-driving cars. An initial attempt towards this goal is to simply share a single network for multi-task learning. We argue that it does not make full use of the fact that detection and segmentation are mutually beneficial. In this paper, we propose a framework called TripleNet to deeply boost these two tasks. On the one hand, to deeply join the two tasks at different scales, triple supervisions including detection-oriented supervision and class-aware/agnostic segmentation supervisions are imposed on each layer of the decoder. Class-agnostic segmentation provides an objectness prior to detection and segmentation. On the other hand, to further intercross the two tasks and refine the features in each scale, two light-weight modules (i.e., the inner-connected module and the attention skip-layer fusion) are incorporated. Because segmentation supervision on each decoder layer are not performed at the test stage and two added modules are light-weight, the proposed TripleNet can run at a real-time speed (16 fps). Experiments on the VOC 2007/2012 and COCO datasets show that TripleNet outperforms all the other one-stage methods on both two tasks (e.g., 81.9% mAP and 83.3% mIoU on VOC 2012, and 37.1% mAP and 59.6% mIoU on COCO) by a single network.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Panoptic Feature Pyramid Networks (2019, background): “the box branch in Mask R-CNN benefits from the mask branch [24], and joint detection and semantic segmentation of thing classes also shows gains [3, 8, 17, 46].”
Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation (2020, background): “With their popularity in the field of natural language processing [8, 39, 43, 49, 60], attention modeling is rapidly adopted in various computer vision tasks, such as image recognition [14, 23, 58, 66, 73], domain adaptation [67, 83], human pose estimation [9, 63, 77], object detection [4] and image”
Efficient Featurized Image Pyramid Network for Single Shot Detector (2019, background): “Generally, deep object detection approaches can ∗ Equal contribution be roughly divided into two categories: two-stage [13, 14, 16, 29] and single-stage detectors [19, 28, 5].”
Learning Rich Features at High-Speed for Single-Shot Object Detection (2019, methodology): “Modern detectors can be divided into two categories: (i) single-stage methods [21, 26, 27, 28, 3, 4] and (ii) two-stage approaches [29, 17, 14, 11, 31].”
Enriched Feature Guided Refinement Network for Object Detection (2019, background): “TripleNet [4] ResNet101 512× 512 - 37.”
Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks (2019, background): “Some others turned to fully convolutional networks (FCNs) [3, 34, 77].”
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation (2020, background): “Different from semantic segmentation [32,10,6,34,39], instance segmentation also differentiates multiple object instances.”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, unknown): “into single-stage [35, 42, 40, 29, 27, 4] and two-stage methods [18, 43, 17, 41, 8, 21].”
Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation (2020, background): “ries, respectively. 2 RELATED WORK ObjectDetectionandInstanceSegmentation.Objectdetection, which plays an important role in various vision applications, has been actively investigated in past decades [4, 6, 7, 23, 32, 34, 40, 41]. With deep learning, two mainstream frameworks, i.e., single-stage det”