Instance-level Human Parsing via Part Grouping Network

By Ke Gong, Xiaodan Liang, Yicheng Li, Yimin Chen, Ming-Hsuan Yang, Liang Lin. European Conference on Computer Vision, 2018

381 citations recorded by the source corpus.

Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass. Several related works all follow the “parsing-by-detection” pipeline that heavily relies on separately trained detection models to localize instances and then performs human parsing for each instance sequentially. Nonetheless, two discrepant optimization targets of detection and parsing lead to suboptimal representation learning and error accumulation for final results. In this work, we make the first attempt to explore a detection-free Part Grouping Network (PGN) for efficiently parsing multiple people in an image in a single pass. Our PGN reformulates instance-level human parsing as two twinned sub-tasks that can be jointly learned and mutually refined via a unified network: (1) semantic part segmentation for assigning each pixel as a human part (e.g., face, arms); (2) instance-aware edge detection to group semantic parts into distinct person instances. Thus the shared intermediate representation would be endowed with capabilities in both characterizing fine-grained parts and inferring instance belongings of each part. Finally, a simple instance partition process is employed to get final results during inference. We conducted experiments on PASCAL-Person-Part dataset and our PGN outperforms all state-of-the-art methods. Furthermore, we show its superiority on a newly collected multi-person parsing dataset (CIHP) including 38,280 diverse images, which is the largest dataset so far and can facilitate more advanced human analysis. The CIHP benchmark and our source code are available at http://sysu-hcp.net/lip/.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f832fdf1fac092b4140bf81d38e6bc6af5c1ea65

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans (2020, methodology): “For each image, we apply [18] to obtain the foreground human mask and set the values of the background image pixels as zero.”
Multi-Garment Net: Learning to Dress 3D People From Images (2019, methodology): “Following [20, 3], we abstract away the appearance information in RGB images and extract semantic garment segmentation [20] to reduce the risk of over-fitting, albeit at the cost of disregarding useful shading signal.”
Learning to Reconstruct People in Clothing From a Single RGB Camera (2019, methodology): “To be able to render synthetic semantic segmentation, we first render the LifeScans subjects from different viewpoints and segment the output with the method of [29].”
Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies (2021, methodology): “We use [22] to obtain the SMPL parameters from the 3D human poses and apply [15] to segment foreground humans.”
Progressive One-shot Human Parsing (2020, positive): “Recent methods for human parsing improve parsing performance from utilizing the body structure priors and class relations (Xiao et al. 2018; Gong et al. 2017; Zhu et al. 2018; Gong et al. 2018; Li et al. 2020; Zhan et al. 2019).”
Semantically Multi-Modal Image Synthesis (2020, positive): “Images are evaluated using well-trained segmentation models Uper-Net101 [44] for ADE20K, DRN-D-105 [48] for Cityscapes, off-the-shelf human parser CIHP [9] for DeepFashion.”
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization (2021, methodology): “We first predict the segmentation map S ∈ LH×W and the pose map P ∈ R3×H×W of the image I by utilizing the pre-trained networks [7, 3], where L is a set of integers indicating the semantic labels.”
ClothFlow: A Flow-Based Model for Clothed Person Generation (2019, methodology): “To obtain the person pose and layout representations used for training our target layout generator, we utilize an off-the-shelf pose estimator [3] and human parser [11].”
Self-Correction for Human Parsing (2019, neutral): “60 PGN [10] 90.”
Single-Shot Freestyle Dance Reenactment (2020, methodology): “To enable diverse generation capabilities in terms of appearance (ethnicity, gender and age), pose, and perspective, we combine the Multi-Human Parsing (MHPv2) [52, 22] and the Crowd Instance-level Human Parsing (CIHP) [17] datasets.”