Deep Human Parsing with Active Template Regression

By Xiaodan Liang, Si Liu, Xiaohui Shen, Jianchao Yang, Luoqi Liu, Jian Dong, Liang Lin, Shuicheng Yan. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015

304 citations recorded by the source corpus.

In this work, the human parsing task, namely decomposing a human image into semantic fashion/body regions, is formulated as an active template regression (ATR) problem, where the normalized mask of each fashion/body item is expressed as the linear combination of the learned mask templates, and then morphed to a more precise mask with the active shape parameters, including position, scale and visibility of each semantic region. The mask template coefficients and the active shape parameters together can generate the human parsing results, and are thus called the structure outputs for human parsing. The deep Convolutional Neural Network (CNN) is utilized to build the end-to-end relation between the input human image and the structure outputs for human parsing. More specifically, the structure outputs are predicted by two separate networks. The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters. For a new image, the structure outputs of the two networks are fused to generate the probability of each label for each pixel, and super-pixel smoothing is finally used to refine the human parsing result. Comprehensive evaluations on a large dataset well demonstrate the significant superiority of the ATR framework over other state-of-the-arts for human parsing. In particular, the F1-score reaches 64.38 percent by our ATR framework, significantly higher than 44.76 percent based on the state-of-the-art algorithm [28].

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/600613a2a3a8a56c9070d427272cbde8fad0d7e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic Object Parsing with Graph LSTM (2016, methodology): “4) We apply the proposed Graph LSTM in semantic object parsing, and demonstrate its superiority through comprehensive comparisons on four challenging semantic object parsing datasets (i.e., PASCAL-Person-Part dataset [20], Horse-Cow parsing dataset [21], ATR dataset [22] and Fashionista dataset [23]”
Progressive One-shot Human Parsing (2020, unknown): “…has made signiﬁcant progress by leveraging the large-scale human parsing datasets (Liang et al. 2015b, 2016; Ruan et al. 2019), the parsing results are restricted to the classes pre-deﬁned in the training set, e.g. 18 classes in ATR (Liang et al. 2015a), and 19 classes in LIP (Liang et al. 2018).”
High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling (2020, methodology): “We collect 82,020 object masks from densely annotated segmentation datasets, including video segmentation [11], semantic segmentation [18], salient object segmentation [27,35,19], and human parsing [28].”
Semantic Object Parsing with Local-Global Long Short-Term Memory (2015, methodology): “Previous works often resort to some post-processing techniques to separately address these complex contextual dependencies, such as, super-pixel smoothing [15], mean field approximation [22] and conditional random field [3][38][33].”
Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound (2016, methodology): “Active shape models are realised with CNNs in [35] via regression of multi-template contributions and object location.”
Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing (2017, background): “Although there exist training sets for special scenarios such as fashion pictures [30, 9, 13, 17] and people in constrained situations (e.”
Learning Semantic Neural Tree for Human Parsing (2019, methodology): “It is worth mentioning that some previous methods [35, 24, 33, 34] use body physical structure information to improve the human parsing accuracy.”
Reversible Recursive Instance-Level Object Segmentation (2015, background): “Recently, beyond the traditional object detection [7][29][5][15][4][30] and semantic segmentation tasks [1][23][2][37][19][18][21][20][14][16], instancelevel object segmentation has attracted much attention [10][11][22][31][36][17].”
Instance-level Human Parsing via Part Grouping Network (2018, background): “However, previous approaches only focus on the single-person parsing task in the simpliﬁed and limited conditions, such as fashion pictures [41,8,18,23,6] with upright poses and diverse daily images [11], and disregard more real-world cases where multiple person instances appear in one image.”
Single Image Dehazing via Multi-scale Convolutional Neural Networks (2016, background): “Recently, CNNs have shown an explosive popularity [12,13,14,15].”
Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark (2018, background): “n analyzing images of humans by providing both pixel-wise understanding and high-level joint structures. Recently, convolutional neural networks (CNNs) have achieved exciting success in human parsing [23], [25], [26] and pose estimation [33], [44]. Nevertheless, as demonstrated in many other problem”
TextureGAN: Controlling Deep Image Synthesis with Texture Patches (2017, methodology): “For clothes, weleveragetheclothesparsinginformationprovidedinthedataset[23, 24].”