Pose2Seg: Detection Free Human Instance Segmentation

By Song-Hai Zhang, Ruilong Li, Xin Dong, Paul L. Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, Shimin Hu. 2018

248 citations recorded by the source corpus.

The standard approach to image instance segmentation is to perform the object detection first, and then segment the object from the detection bounding-box. More recently, deep learning methods like Mask R-CNN perform them jointly. However, little research takes into account the uniqueness of the "human" category, which can be well defined by the pose skeleton. Moreover, the human pose skeleton can be used to better distinguish instances with heavy occlusion than using bounding-boxes. In this paper, we present a brand new pose-based instance segmentation framework for humans which separates instances based on human pose, rather than proposal region detection. We demonstrate that our pose-based framework can achieve better accuracy than the state-of-art detection-based approach on the human instance segmentation problem, and can moreover better handle occlusion. Furthermore, there are few public datasets containing many heavily occluded humans along with comprehensive annotations, which makes this a challenging problem seldom noticed by researchers. Therefore, in this paper we introduce a new benchmark "Occluded Human (OCHuman)", which focuses on occluded humans with comprehensive annotations including bounding-box, human pose and instance masks. This dataset contains 8110 detailed annotated human instances within 4731 images. With an average 0.67 MaxIoU for each person, OCHuman is the most complex and challenging dataset related to human instance segmentation. Through this dataset, we want to emphasize occlusion as a challenging problem for researchers to study.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

iShape: A First Step Towards Irregular Shape Instance Segmentation (2021, neutral): “Instead, in the scenario of human detection and segmentation, the OC human [19] and the Crowd Human [28] introduce datasets with larger MaxIoU.”
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation (2022, methodology): “To evaluate the performance of human pose estimation models on the human instances with heavy occlusion, we test the ViTPose variants and representative models on the OCHuman val and test set with ground truth bounding boxes.”
Real-Time High-Resolution Background Matting (2020, background): “Specific applications of human segmentation and parsing have also received considerable attention in recent works [33, 19].”