Deep High-Resolution Representation Learning for Human Pose Estimation
By Ke Sun, Bin Xiao, Dong Liu, Jingdong Wang. 2019
5,208 citations recorded by the source corpus.
In this paper, we are interested in the human pose estimation problem with a focus on learning reliable high-resolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process. We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich high-resolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection dataset and the MPII Human Pose dataset. In addition, we show the superiority of our network in pose tracking on the PoseTrack dataset. The code and models have been publicly available at https://github.com/leoxiaobin/deep-high-resolution-net.pytorch.
Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dacheng Tao
- Jianfeng Gao
- Luc Van Gool
- Mohit Bansal
- Quoc V. Le
- Jiashi Feng
- Trevor Darrell
- Alan L. Yuille
In Biomedical Engineering
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Yi Yang
- Alan L. Yuille
In 3D Vision & Pose
- Michael J. Black
- Jitendra Malik
- Jun Liu
- Andrea Vedaldi
- Kyoung Mu Lee
- Gyeongsik Moon
- Luc Van Gool
- David Novotný
Citations in the corpus
- Grounded Language-Image Pre-training (2021, background): “However, to gain fine-grained understanding of images, as required by many tasks, such as object detection [33, 46], segmentation [7,37], human pose estimation [51,58], scene understanding [15, 27, 59], action recognition [19], visionlanguage understanding [8,29–32,38,50,52,65,67], objectlevel visua”
- DetNAS: Backbone Search for Object Detection (2019, background): “The importance of backbones is also demonstrated in DetNet [11], Deformable ConvNets v2 [29], ThunderNet [21] in real-time object detection, and HRNet [24] in keypoint detection.”
- Albumentations: fast and flexible image augmentations (2018, methodology): “For this study, we used well-known UNet-based model architecture [67], with HRNetV2 encoder [68].”
- MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, background): “• HRNet [30, 31]: a new backbone with a focus on learning reliable high-resolution representations, proposed in 2019.”
- Deep High-Resolution Representation Learning for Visual Recognition (2019, methodology): “(1) We extend the network (named as HRNetV 1 ) proposed in [130], to two versions: HRNetV 2 and HRNetV 2 p, which explore all the four-resolution representations.”
- Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, methodology): “antic segmentation head proposed in HRNet [70]. effective model could establish a solid baseline and further benefit the research community. A. HRNet Variant We introduce our modifications to the HRNet [66,70] that are used in our ensemble model for Mapillary Vistas. All hyper-parameters for training ”
- Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding (2021, background): “But its quadratic increase of both computational and memory complexity hinders its application on many vision tasks that require highresolution feature maps computed on high-resolution images1, like object detection [31, 23], segmentation [25, 5], and human pose estimation [45, 34].”
- Vision Transformers for Dense Prediction (2021, background): “More recent architectures maintain a high-resolution representation together with multiple lower-resolution representations throughout the network [39, 44].”
- CenterNet: Keypoint Triplets for Object Detection (2019, methodology): “We also replace the backbone with HRNetW64 [40, 41], which reports APs of 44.”
- Focal Self-attention for Local-Global Interactions in Vision Transformers (2021, neutral): “9 HRNet-w48 [54] OCRNet [77] 71M 664G 45.”
- Conditional DETR for Fast Training Convergence (2021, background): “In the future, we will study the proposed conditional cross-attention mechanism for human pose estimation [8, 41, 36] and line segment detection [43].”
- Res2Net: A New Multi-Scale Backbone Architecture (2019, background): “One common operation in [5], [9], [11], [48], [49] is that they all use pooling or up-sample to re-size the feature map to 2 times of the original scale to save the computational budget while maintaining or even improving performance.”
- AANet: Adaptive Aggregation Network for Efficient Stereo Matching (2020, methodology): “Although our CSA module is similar to HRNet[32], they have two major differences.”
- MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video (2022, methodology): “Utilizing more powerful 2D detector HRNet [36], our model further improves roughly 4.5mm (10.2”
- Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes (2021, methodology): “But due to its deep multiple branches and multiple repeated fusion of multi-scale feature [13], HRNet seems not to be an ideal choice for real-time semantic segmentation.”
- Dynamic ReLU (2020, methodology): “Testing: We use the person detectors provided by [38] and follow the evaluation procedure in [38,31].”