DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch

By Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu, Raquel Urtasun. IEEE International Conference on Computer Vision, 2019

310 citations recorded by the source corpus.

Our goal is to significantly speed up the runtime of current state-of-the-art stereo algorithms to enable real-time inference. Towards this goal, we developed a differentiable PatchMatch module that allows us to discard most disparities without requiring full cost volume evaluation. We then exploit this representation to learn which range to prune for each pixel. By progressively reducing the search space and effectively propagating such information, we are able to efficiently compute the cost volume for high likelihood hypotheses and achieve savings in both memory and computation.Finally, an image guided refinement module is exploited to further improve the performance. Since all our components are differentiable, the full network can be trained end-to-end. Our experiments show that our method achieves competitive results on KITTI and SceneFlow datasets while running in real-time at 62ms.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/67fc98119edf2debefd3a533a374429b1836aa10

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “Meanwhile, we also design a real-time version of ACVNet, named ACVNet-Fast, which outperforms stateof-the-art real-time methods [4, 10, 20, 22].”
PatchmatchNet: Learned Multi-View Patchmatch Stereo (2020, background): “DeepPruner [11] develops a differentiable Patchmatch module, without learnable parameters, to discard most disparities and then builds a lightweight cost volume, which is regularized by a 3D CNN.”
Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching (2019, methodology): “DeepPruner [5] is a coarse to fine method which proposes a differentiable PatchMatch-based module to predict the pruned search range for each pixel.”
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (2021, methodology): “Following [7], we combine KITTI 2012 and KITTI 2015 image pairs for the evaluation of KITTI 2015 while only use KITTI 2012 image pairs for the evaluation of KITTI 2012.”
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching (2020, unknown): “74 400 Deep-Pruner[8] 0.”
AANet: Adaptive Aggregation Network for Efficient Stereo Matching (2020, methodology): “The very recent work, DeepPruner [6], shares a similar goal with us to build efficient stereo models.”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “48 DeepPruner (Fast) [83] Supervised 3D 3.”
Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, background): “In Table 1, we can observe that LEAStereo achieves the best performance using only near one third of parameters in comparison to the SOTA hand-crafted methods (e.g., [6]).”