CRF-CNN: Modeling Structured Information in Human Pose Estimation

By Xiao Chu, Wanli Ouyang, Hongsheng Li, Xiaogang Wang. 2016

81 citations recorded by the source corpus.

Deep convolutional neural networks (CNN) have achieved great success. On the other hand, modeling structural information has been proved critical in many vision problems. It is of great interest to integrate them effectively. In a classical neural network, there is no message passing between neurons in the same layer. In this paper, we propose a CRF-CNN framework which can simultaneously model structural information in both output and hidden feature layers in a probabilistic way, and it is applied to human pose estimation. A message passing scheme is proposed, so that in various layers each body joint receives messages from all the others in an efficient way. Such message passing can be implemented with convolution between features maps in the same layer, and it is also integrated with feedforward propagation in neural networks. Finally, a neural network implementation of end-to-end learning CRF-CNN is provided. Its effectiveness is demonstrated through experiments on two benchmark datasets.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Structured Convolutional Kernel Networks for Airline Crew Scheduling (2021, methodology): “, 2016) and CNN-CRF (standard two-layer) (Chu et al., 2016) yield comparable results (4.”
Reasoning Visual Dialogs With Structural and Partial Observations (2019, background): “Another stream is to build graphically structured neural networks to approximate the learning/inference process of graphical models [30, 55, 29, 14, 4, 17, 60, 8].”
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction (2017, methodology): “In order to provide an answer and inspired by recent works exploiting graphical models within deep learning architectures [5, 40], we introduce Attention-Gated Conditional Random Fields (AG-CRFs), which allow to learn robust feature map representations at each scale by exploiting the information…”
Multi-context Attention for Human Pose Estimation (2017, methodology): “graph models, e.g., mixture of body parts [45,8,30] or pictorial structures [31]. Recently, signiﬁcant progresses have been achieved by introducing ConvNets for learning better feature representation [39,38,37,8,40,43,9,10,33,29]. For example, Chen and Yuille [8] introduced the ConvNet to learn both”
Spatial As Deep: Spatial CNN for Traffic Scene Understanding (2017, methodology): “Researchers also attempted to combine CNN with graphical models like MRF or CRF, in which message pass is realized by convolution with large kernels (Liu et al. 2015; Tompson et al. 2014; Chu et al. 2016).”
Few-Shot Object Recognition from Machine-Labeled Web Images (2016, background): “We will explore the structural storage of external memory for different vision datasets, or for different tasks, e.g ., image classiﬁcation [15], object detection [29], human pose estimation [7], and learn task-speciﬁc representation on demand in our abstraction memory framework.”