Pyramid Stereo Matching Network
By Jia-Ren Chang, Yong-Sheng Chen. 2018
1,823 citations recorded by the source corpus.
Recent work has shown that depth estimation from a stereo pair of images can be formulated as a supervised learning task to be resolved with convolutional neural networks (CNNs). However, current architectures rely on patch-based Siamese networks, lacking the means to exploit context information for finding correspondence in ill-posed regions. To tackle this problem, we propose PSMNet, a pyramid stereo matching network consisting of two main modules: spatial pyramid pooling and 3D CNN. The spatial pyramid pooling module takes advantage of the capacity of global context information by aggregating context in different scales and locations to form a cost volume. The 3D CNN learns to regularize cost volume using stacked multiple hourglass networks in conjunction with intermediate supervision. The proposed approach was evaluated on several benchmark datasets. Our method ranked first in the KITTI 2012 and 2015 leaderboards before March 18, 2018. The codes of PSMNet are available at: https://github.com/JiaRenChang/PSMNet.
Subfields: Computer Vision and Pattern Recognition, Media Technology, Model Architectures, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Jitendra Malik
- Ming-Hsuan Yang
- Trevor Darrell
- Bernt Schiele
- Andrea Vedaldi
In Media Technology
- Yuchao Dai
- Jia Deng
- Zachary Teed
- Peng Wang
- Andreas Geiger
- Ping Tan
- Nantheera Anantrasirichai
- D. Bull
In Model Architectures
- Philip H. S. Torr
- Xiaogang Wang
- Raquel Urtasun
- Jiaya Jia
- Wanli Ouyang
- Dacheng Tao
- Ping Luo
- Hengshuang Zhao
In 3D Vision & Pose
- Andreas Geiger
- Marc Pollefeys
- Michael Firman
- Jia Deng
- Zachary Teed
- Mohammed Bennamoun
- Raquel Urtasun
- Xiaogang Wang
Citations in the corpus
- IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, background): “Recent advances [1], [2], [3], [4], [5], [6] in these vision tasks have shown that deep neural network (DNN) can significantly improve the estimation quality.”
- BADGR: An Autonomous Self-Supervised Learning-Based Navigation System (2020, neutral): “Oneapproachtoimprove uponSLAMmethodsistodirectlyestimatethegeometryofthe scene[9],[10].”
- Efficient Attention: Attention with Linear Complexities (2018, methodology): “Table 8 compares EA-PSMNet with other state-of-the-art approaches and shows that it substantially outperforms all competing methods.”
- Deep learning in optical metrology: a review (2022, methodology): “Chang et al.390 developed a pyramid stereomatching network (PSMNet) to enhance the matching accuracy by using the 3D CNN-based spatial pyramid pooling and multiple hourglass networks.”
- Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “As shown in Table 3, the EPE is reduced by 39.5% for GwcNet [7], 42.2% for PSMNet [2] and 14.4% for CFNet [16].”
- Factorized Attention: Self-Attention with Linear Complexities (2018, methodology): “As table 6 shows, the proposed FA-PSMNet demonstrated significant improvement over the highly competitive baseline, which already surpassed the former state-of-the-art by a substantial margin.”
- Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving (2019, methodology): “In order to study the impact of quality of depth maps on the overall performance of proposed method, we implemented four different depth generators [10, 16, 22, 2].”
- Learning Depth-Guided Convolutions for Monocular 3D Object Detection (2019, methodology): “To study the impact of accuracy of depth maps on the performance of our method, we extract depth maps using four different methods [13, 10, 35, 3] and then apply them to 3D detection.”
- HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching (2020, methodology): “These features encode multi-scale details of the image, similar to [7] (Sec.”
- Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, methodology): “Different designs of the feature net and the matching net form variants of the volumetric networks [4, 10, 11, 12, 6].”
- Stereo R-CNN Based 3D Object Detection for Autonomous Driving (2019, background): “Note that our task is different with pixel-wise disparity estimation problem where the result might encounter either discontinuity at ill-posed regions (SGM [10]), or oversmooth at edge areas (CNN based methods [29, 12, 2]).”
- Real-Time Self-Adaptive Deep Stereo (2018, methodology): “Thereby, [21] ignited the spread of end-to-end stereo architectures [17, 24, 19, 4, 16, 11] that quickly outmatched any other technique on the KITTI benchmarks by leveraging on a peculiar training protocol.”
- Image Matching from Handcrafted to Deep Features: A Survey (2020, neutral): “D convolutional strategy has been introduced in Chang and Chen (2018).”
- Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, methodology): “Recent binocular stereo methods [3, 5] use a similar cost volume approach, but avoid frame selection by using a fixed baseline stereo pair.”
- Unifying Flow, Stereo and Depth Estimation (2022, methodology): “Unlike previous representative stereo networks [19], [24], [25] that usually rely on a predefined disparity range (typically 192 pixels) to construct the local cost volume, our method is more flexible and can support unconstrained disparity prediction.”
- Scale-Space Flow for End-to-End Optimized Video Compression (2020, background): “[6, 18, 13]) as well as the use of multi-scale pyramids in deep flow estimation networks [28, 14].”
- Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference (2019, background): “Scalable MVS Reconstruction The memory requirement of learned cost volume regularizations [14, 15, 13, 5, 30] grows cubically with the model resolution, which will be intractable when large image sizes or wide depth ranges occur.”
- GA-Net: Guided Aggregation Net for End-To-End Stereo Matching (2019, methodology): “In GC-Net [13] and PSMNet [3], K = 3, C = 32,64 or 128 and in our GA-Nets, K is used as 5 (for SGA layer).”
- RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching (2021, methodology): “Many variations on this design have been proposed, such as using a stacked 3D hourglass to process the cost volume[4], or designing new aggregation layers to better propagate information[47].”