Deep Surface Normal Estimation With Hierarchical RGB-D Fusion

By Jin Zeng, Yanfeng Tong, Yunmu Huang, Qiong Yan, Wenxiu Sun, Jing Chen, Yongtian Wang. 2019

77 citations recorded by the source corpus.

The growing availability of commodity RGB-D cameras has boosted the applications in the field of scene understanding. However, as a fundamental scene understanding task, surface normal estimation from RGB-D data lacks thorough investigation. In this paper, a hierarchical fusion network with adaptive feature re-weighting is proposed for surface normal estimation from a single RGB-D image. Specifically, the features from color image and depth are successively integrated at multiple scales to ensure global surface smoothness while preserving visually salient details. Meanwhile, the depth features are re-weighted with a confidence map estimated from depth before merging into the color branch to avoid artifacts caused by input depth corruption. Additionally, a hybrid multi-scale loss function is designed to learn accurate normal estimation given noisy ground-truth dataset. Extensive experimental results validate the effectiveness of the fusion strategy and the loss design, outperforming state-of-the-art normal estimation schemes.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, methodology): “calculate the angle between them and take the mean value and the median value. We then compute the fraction of pixels of which the angle error is less than t, where t= 11:25;22:5;30, as adopted in [26]. 3) Training Implementation: We implemented DispNetCSS and DispNormNet using PyTorch. All the mode”
Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation (2019, background): “Surface normal estimation is a popular problem tackled by deep convolutional networks [51, 13, 60, 57].”
Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, background): “First, we note that this design is reminiscent of fusion architectures in multimodal networks, where data from different modalities are processed and combined in a neural network [24, 78].”
Deep Multimodal Fusion by Channel Exchanging (2020, methodology): “The aggregation can be implemented as averaging [18], concatenation [50], and self-attention [44].”