Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching

By Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong Tan, Ping Tan. Computer Vision and Pattern Recognition, 2019

959 citations recorded by the source corpus.

The deep multi-view stereo (MVS) and stereo matching approaches generally construct 3D cost volumes to regularize and regress the output depth or disparity. These methods are limited when high-resolution outputs are needed since the memory and time costs grow cubically as the volume resolution increases. In this paper, we propose a both memory and time efficient cost volume formulation that is complementary to existing multi-view stereo and stereo matching approaches based on 3D cost volumes. First, the proposed cost volume is built upon a standard feature pyramid encoding geometry and context at gradually finer scales. Then, we can narrow the depth (or disparity) range of each stage by the depth (or disparity) map from the previous stage. With gradually higher cost volume resolution and adaptive adjustment of depth (or disparity) intervals, the output is recovered in a coarser to fine manner. We apply the cascade cost volume to the representative MVS-Net, and obtain a 35.6% improvement on DTU benchmark (1st place), with 50.6% and 59.3% reduction in GPU memory and run-time. It is also the state-of-the-art learning-based method on Tanks and Temples benchmark. The statistics of accuracy, run-time and GPU memory on other representative stereo CNNs also validate the effectiveness of our proposed method. Our source code is available at https://github.com/alibaba/cascade-stereo.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f88fb3a3ef31ecd9a00c7bd800cbd8f9bdb2f704

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Unifying Flow, Stereo and Depth Estimation (2022, methodology): “A popular multi-view depth pipeline is using the plane-sweep stereo [72], [79] approach, where different depth planes are tested for correctness.”
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images (2024, methodology): “Despite the conceptual similarity with the well-established MVS reconstruction pipeline [13,31,48], our approach possesses unique advantages.”
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting (2023, background): “In particular, Reiser et al. [38] accelerate NeRF’s training by splitting a big MLP into thousands of tiny MLPs. MVSNeRF [8] constructs a 3D cost volume [19, 56] and renders high-quality images from novel viewpoints.”
NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video (2021, background): “Some recent works [12, 4] improve this pipeline with a coarse-to-fine approach.”
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (2021, methodology): “While it is noteworthy that the insight of cascade cost volume has also been investigated in [11], our work differs from theirs in the following three main aspects: First, casstereo constructs higher-resolution sparse cost volume to predict an initial disparity estimation.”
Space-time Neural Irradiance Fields for Free-Viewpoint Video (2020, background): “Existing multi-view stereo (MVS) methods (either geometric-based [60, 18] or learning-based [24, 72, 20, 37]) assume static scene and thus not suitable for dynamic videos as it often produces erroneous depth for moving objects or untextured regions.”
Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “We apply the two-stage cascaded method proposed by [6] to GwcNet, the corresponding model is denoted as Gwc-CAS.”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “Based on MVSNet [34], RMVSNet [35] uses GRU to reduce memory cost, MVSCRF [36] adds CRF module, cascade MVSNet [37] Change MVSNet to cascade structure.”
DUSt3R: Geometric 3D Vision Made Easy (2023, neutral): “The fully handcrafted [32, 34, 107, 147, 175], the more recent scene optimization based [31, 71, 76, 77, 143, 146, 148, 163], or learning based [40, 53, 65, 86, 161, 164, 180] approaches all depend on camera parameter estimates obtained via complex calibration procedures, either during the data…”
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting (2024, methodology): “Recently, deep learning-based MVS methods have also been proposed [16,25,53,69,70], offering efficient high-quality reconstruction in a feed-forward manner.”
IExpressNet: Facial Expression Recognition with Incremental Classes (2020, background): “Recently, with the extensive application of deep learning in the field of computer vision [20, 23], end-to-end feature extraction is increasingly applied in FER.”
Neural 3D Video Synthesis from Multi-view Video (2021, neutral): “[20] Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong”
VGGT: Visual Geometry Grounded Transformer (2025, neutral): “…from multiple overlapping images, typically assuming known camera parameters, which are often estimated with SfM. MVS methods can be divided into three categories: traditional handcrafted [26, 27, 64, 88], global optimization [25, 50, 91, 100], and learning-based meth-ods [30, 49, 55, 99, 106].”
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo (2021, methodology): “Following works have extended this technique with recurrent plane sweeping [51], point-based densification [8], confidence-based aggregation [30], and multiple cost volumes [10, 18], improving the reconstruction quality.”
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers (2020, background): “Following the same idea of computing matching cost by learnt 3D convolutions, other works [40, 7, 41, 15] attempt to enable high resolution inference, mitigate the memory constraint, and/or leverage richer context information via a multi-resolution approach.”