Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers
By Zhaoshuo Li, Xingtong Liu, Nathan Drenkow, Francis X. Creighton, Russell H. Taylor, Mathias Unberath. IEEE International Conference on Computer Vision, 2020
394 citations recorded by the source corpus.
Stereo depth estimation relies on optimal correspondence matching between pixels on epipolar lines in the left and right images to infer depth. In this work, we revisit the problem from a sequence-to-sequence correspondence perspective to replace cost volume construction with dense pixel matching using position information and attention. This approach, named STereo TRansformer (STTR), has several advantages: It 1) relaxes the limitation of a fixed disparity range, 2) identifies occluded regions and provides confidence estimates, and 3) imposes uniqueness constraints during the matching process. We report promising results on both synthetic and real-world datasets and demonstrate that STTR generalizes across different domains, even without fine-tuning.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/75bcdf04827cd230d88b70c622dce8868eb50a3b
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Luc Van Gool
- Xiaogang Wang
- Yi Yang
- Jitendra Malik
- Ming-Hsuan Yang
- Trevor Darrell
- Bernt Schiele
Citations in the corpus
- LoFTR: Detector-Free Local Feature Matching with Transformers (2021, methodology): “Concurrently with our work, [20] proposes to use Transformer for disparity estimation.”
- Unifying Flow, Stereo and Depth Estimation (2022, background): “However, STTR relies on a complex optimal transport matching layer and doesn’t produce predictions for occluded pixels, while we use a much simpler softmax operation and a simple flow propagation layer to handle occlusions.”
- GMFlow: Learning Optical Flow via Global Matching (2021, methodology): “For dense correspondences, STTR [22] is a Transformer-based method for stereo matching, which can be viewed as a special case of optical flow.”
- Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, background): “STTR [32] and LSTR [34] adopt transformer for disparity estimation and lane shape prediction respectively.”
- BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, background): “[39] fuse single-view images with multi-view images and introduce depth-sampling to reduce the cost of computation.”
- Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation (2022, background): “One line of networks [12, 21, 22, 27, 29, 44, 49] only uses 2D convolutions.”