Neural Window Fully-connected CRFs for Monocular Depth Estimation

By Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, Ping Tan. Computer Vision and Pattern Recognition, 2022

295 citations recorded by the source corpus.

Estimating the accurate depth from a single image is challenging since it is inherently ambiguous and ill-posed. While recent works design increasingly complicated and powerful networks to directly regress the depth map, we take the path of CRFs optimization. Due to the expensive computation, CRFs are usually performed between neighborhoods rather than the whole graph. To leverage the potential of fully-connected CRFs, we split the input into windows and perform the FC-CRFs optimization within each window, which reduces the computation complexity and makes FC-CRFs feasible. To better capture the relationships between nodes in the graph, we exploit the multi-head attention mechanism to compute a multi-head potential function, which is fed to the networks to output an optimized depth map. Then we build a bottom-up-top-down structure, where this neural window FC-CRFs module serves as the decoder, and a vision transformer serves as the encoder. The experiments demonstrate that our method significantly improves the performance across all metrics on both the KITTI and NYUv2 datasets, compared to previous methods. Furthermore, the proposed method can be directly applied to panorama images and outperforms all previous panorama methods on the MatterPort3D dataset.11Project page: https://weihaosky.github.io/newcrfs

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/daee1a8ee8fc6b73812ae854716a87d5db962716

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

UniDepth: Universal Monocular Metric Depth Estimation (2024, methodology): “While existing MMDE methods [2, 13, 15, 33, 34, 36, 50] have demonstrated remarkable accuracy across different benchmarks, they require training and testing on datasets with similar camera intrinsics and scene scales.”
BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation (2022, methodology): “BinsFormer achieves very competitive results, including NeWCRFs [53] and VA-DepthNet [54], which are the SoTA Swin backbones for MDE.”
DUSt3R: Geometric 3D Vision Made Easy (2023, unknown): “Our method does not reach the accuracy levels of the best methods.”