Deep High-Resolution Representation Learning for Visual Recognition

By Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang and 2 more. 2019

4,876 citations recorded by the source corpus.

High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at https://github.com/HRNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, neutral): “8 Panoptic-DeepLab (HRNet-W48 [81]) [18] 3 71.”
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs (2021, positive): “We choose the following three recent pedestrian detection models trained on CityPerson (available in [6]): (i) Faster-RCNN [10] (ii) Cascade-RCNN [1] with the HRNet [11] backbone, and (iii) CSP-ResNet-50 [9].”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, background): “Since then, deeper and more effective con- 2While there are efficient methods to implement a sliding-window based convolution layer on general hardware, thanks to its shared kernel weights across a feature map, it is difficult for a sliding-window based self-attention layer to have efficient memory ”
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, methodology): “HRFormer [48] uses multi-resolution parallel design [42] to learn high-resolution representations.”
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, methodology): “We use ResNet-18 backbone [20] and HRNet head [74] as our base model for all of our experiments.”
FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “We select the current SOTA methods HRNet+OCR [38, 46] and SegFormer [42] as representatives.”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, background): “The second backbone is HRNet+OCR [10, 11], a relatively new promising architecture specially designed to produce highresolution output.”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, methodology): “BiSeNet (Yu et al., 2018) and HRNet (Wang et al., 2019) design multiple paths to learn both high-and low-resolution representations for better dense prediction.”
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, background): “attention module [9] and multiple scales [44], [45].”
Vision Transformers for Dense Prediction (2021, background): “More recent architectures maintain a high-resolution representation together with multiple lower-resolution representations throughout the network [39, 44].”
Contextual Transformer Networks for Visual Recognition (2021, background): “HRNet [26] maintains high-resolution representations through the whole process by connecting the high-to-low resolution convolution streams in parallel and meanwhile exchanging the information across resolutions repeatedly, pursuing not only semantically strong but also spatially precise representat”
Exploring Cross-Image Pixel Contrast for Semantic Segmentation (2021, methodology): “Such memory design allows us to access more representative data samples during each training step and fully explore structural relations between pixels and semantic-level segHRNet-W48 [65]”
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision (2021, methodology): “We report the additional results on HRNet in Table 3.”
Conformer: Local Features Coupling Global Representations for Visual Recognition (2021, background): “Unfortunately, most CNNs [29, 37, 19, 39, 48, 23, 44] are good at extracting local features but experience difficulty to capture global cues.”
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs (2022, background): “…depth-wise convolutions into conventional networks, whose sizes range from 3 × 3 to 31 × 31, although there exist other alternatives to introduce large receptive ﬁelds via a single or a few layers, e.g . feature pyramids [96], dilated convolutions [14, 106, 107] and deformable convolutions [24].”
CBNet: A Composite Backbone Network Architecture for Object Detection (2021, methodology): “Besides, CBNet is compatible with feature enhancing networks [21], [22] and model ensemble method [23].”