PatchmatchNet: Learned Multi-View Patchmatch Stereo

By Fangjinhua Wang, Silvano Galliani, Christoph Vogel, Pablo Speciale, Marc Pollefeys. Computer Vision and Pattern Recognition, 2020

460 citations recorded by the source corpus.

We present PatchmatchNet, a novel and learnable cascade formulation of Patchmatch for high-resolution multi-view stereo. With high computation speed and low memory requirement, PatchmatchNet can process higher resolution imagery and is more suited to run on resource limited devices than competitors that employ 3D cost volume regularization. For the first time we introduce an iterative multi-scale Patchmatch in an end-to-end trainable architecture and improve the Patchmatch core algorithm with a novel and learned adaptive propagation and evaluation scheme for each iteration. Extensive experiments show a very competitive performance and generalization for our method on DTU, Tanks & Temples and ETH3D, but at a significantly higher efficiency than all existing top-performing models: at least two and a half times faster than state-of-the-art methods with twice less memory usage. Code is available at https://github.com/FangjinhuaWang/PatchmatchNet.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b9ec0bb70a2425493f187ccaf8ea0461e90a7381

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, unknown): “In response, several approaches aim to enhance reconstruction completeness and accuracy by integrating deep neural networks into the matching process [41], [42].”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, background): “One typical approach is transforming image-based depth maps to pseudo-LiDAR to mimic the LiDAR signal (Wang et al. 2019; You et al. 2019; Qian et al. 2020).”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “…Furu [31] 0.613 0.941 0.777 Tola [90] 0.342 1.190 0.766 Gipuma [32] 0.283 0.873 0.578 MVSNet [110] 0.396 0.527 0.462 CVP-MVSNet [109] 0.296 0.406 0.351 UCS-Net [17] 0.338 0.349 0.344 CER-MVS [55] 0.359 0.305 0.332 CIDER [107] 0.417 0.437 0.427 PatchmatchNet [99] 0.427 0.277 0.352 GeoMVSNet [119] 0”
DUSt3R: Geometric 3D Vision Made Easy (2023, unknown): “…the fruits of decades of advances in various sub-fields such as keypoint detection [26, 28, 63, 97] and matching [10, 60, 100, 120], robust estimation [3, 10, 181], Structure-from-Motion (SfM) and Bundle Adjustment (BA) [20, 59, 106], dense Multi-View Stereo (MVS) [40, 107, 139, 158, 176], etc.”
Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, background): “Following the 4D combined cost volume, cascade cost volumes [6,16,19] further reduce the memory and computational complexity of cost volume construction by building a cost volume pyramid in a coarse-to-fine manner to progressively narrow down the target disparity range and refine the depth map.”