High-Resolution Representations for Labeling Pixels and Regions

By Ke Sun, Yang Zhao, Borui Jiang, Tianheng Cheng, Bin Xiao, Dong Liu, Yadong Mu, Xinggang Wang, Wenyu Liu, Jingdong Wang. 2019

909 citations recorded by the source corpus.

High-resolution representation learning plays an essential role in many vision problems, e.g., pose estimation and semantic segmentation. The high-resolution network (HRNet)~\cite{SunXLW19}, recently developed for human pose estimation, maintains high-resolution representations through the whole process by connecting high-to-low resolution convolutions in \emph{parallel} and produces strong high-resolution representations by repeatedly conducting fusions across parallel convolutions. In this paper, we conduct a further study on high-resolution representations by introducing a simple yet effective modification and apply it to a wide range of vision tasks. We augment the high-resolution representation by aggregating the (upsampled) representations from all the parallel convolutions rather than only the representation from the high-resolution convolution as done in~\cite{SunXLW19}. This simple modification leads to stronger representations, evidenced by superior results. We show top results in semantic segmentation on Cityscapes, LIP, and PASCAL Context, and facial landmark detection on AFLW, COFW, $300$W, and WFLW. In addition, we build a multi-level representation from the high-resolution representation and apply it to the Faster R-CNN object detection framework and the extended frameworks. The proposed approach achieves superior results to existing single-model networks on COCO object detection. The code and models have been publicly available at \url{this https URL}.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

MMDetection: Open MMLab Detection Toolbox and Benchmark (2019, background): “• HRNet [30, 31]: a new backbone with a focus on learning reliable high-resolution representations, proposed in 2019.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “…represents a very substantial extension of our previous conference paper [130] with an additional material added from our unpublished technical report [131] as well as more object detection results under recently-developed start-of-the-art object detection and instance segmentation frameworks.”
Object-Contextual Representations for Semantic Segmentation (2019, methodology): “We evaluate the performance over 59 categories following [48].”
CenterNet: Keypoint Triplets for Object Detection (2019, methodology): “We also replace the backbone with HRNetW64 [40, 41], which reports APs of 44.”
MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, methodology): “We use the HRNet-W48 [37] architecture as our model.”
Res2Net: A New Multi-Scale Backbone Architecture (2019, methodology): “k to learn high-frequency residuals by using the up-sampled low-resolution features learned by a lowresolution network. Other than the low-resolution representations in current works, the HRNet [48], [49] introduces high-resolution representations in the network and repeatedly performs multiscale fu”
Learning Dynamic Routing for Semantic Segmentation (2020, background): “In particular, traditional human-designed networks as well as searched architectures, including FCN-32s [24], U-Net [28], DeepLabV3 [5], HRNetV2 [32], and AutoDeepLab [22], are modeled in the routing space with similar connection patterns, as visualized in Fig.”
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation (2019, background): “UNet++ has since been quickly adopted by the research community, either as a strong baseline for comparison [52]–[55], or as a source of inspiration for developing newer semantic segmentation architectures [56]–[61]; it has also been utilized for multiple applications, such as segmenting objects in ”
Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review-Part I: Evolution and Recent Trends (2020, methodology): “The described advances, such as the extraction of features from multiple scales, the usage of contextual information from global and local scales and relying completely on a CNN model without combining it with graphical models for refinement, were aggregated in DeepLabV3 [59] (see Figure 11, right).”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, methodology): “Most of the existing works mainly addressed semantic segmentation through (i) increasing the resolution of feature maps [12,13,52], (ii) constructing more reliable context information [61,60,23,27] and (iii) exploiting boundary information [5,9,42,53].”
Whole-Body Human Pose Estimation in the Wild (2020, methodology): “The results are shown in Table 3a. HR-Ours is our implementation of HRNetV2-W18 [54] (HR).”
An Empirical Study of Remote Sensing Pretraining (2022, methodology): “On the other hand, the IMP-Swin-T performs to be competitive and the IMP-ViTAEv2-S achieves the best performance on the iSAID dataset, outperforming the SOTA methods such as the HRNet [97] and OCR [100] as well as a series of methods that specially designed for aerial semantic segmentation, e.g., th”
Face X-Ray for More General Face Forgery Detection (2019, methodology): “For fair comparison, we also show the results of the binary classi-ﬁer using the same network architecture with ours, which is denoted as HRNet in the table.”
Scaling Wide Residual Networks for Panoptic Segmentation (2020, neutral): “7 HRNetV2 + OCR + SegFix [99, 100, 79] C, MV - - 84.”