GA-Net: Guided Aggregation Net for End-To-End Stereo Matching

By Feihu Zhang, Victor Adrian Prisacariu, Ruigang Yang, Philip H. S. Torr. 2019

802 citations recorded by the source corpus.

In the stereo matching task, matching cost aggregation is crucial in both traditional methods and deep neural network models in order to accurately estimate disparities. We propose two novel neural net layers, aimed at capturing local and the whole-image cost dependencies respectively. The first is a semi-global aggregation layer which is a differentiable approximation of the semi-global matching, the second is the local guided aggregation layer which follows a traditional cost filtering strategy to refine thin structures. These two layers can be used to replace the widely used 3D convolutional layer which is computationally costly and memory-consuming as it has cubic computational/memory complexity. In the experiments, we show that nets with a two-layer guided aggregation block easily outperform the state-of-the-art GC-Net which has nineteen 3D convolutional layers. We also train a deep guided aggregation network (GA-Net) which gets better accuracies than state-of-the-art methods on both Scene Flow dataset and KITTI benchmarks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SOCRATES: Introducing Depth in Visual Wildlife Monitoring Using Stereo Vision (2022, neutral): “43px, which are in line with prior work on comparable scenes [5,6].”
IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, background): “Recent advances [1], [2], [3], [4], [5], [6] in these vision tasks have shown that deep neural network (DNN) can significantly improve the estimation quality.”
Deep learning in optical metrology: a review (2022, background): “Then the outputs of CNN are used to obtain a high-accuracy absolute phase for further 3D reconstruction. b The input color fringe pattern of a David plaster model. c The 3D reconstruction result of the color-coded approach proposed by Zhang et al.301. d The 3D reconstruction result of our deep-learn”
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers (2020, methodology): “Under multiple evaluation settings, we compare STTR with prior work spanning the primary learning-based stereo depth paradigms, including correlation-based AANet [39], 3D convolution-based PSMNet [5] and GANet-11 [43], correlation and 3D convolution hybrid approach GwcNet-g [16] and a classification”
Learning to Recover 3D Scene Shape from a Single Image (2020, methodology): “We use the GA-Net [53] method to compute the disparity for supervision.”
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching (2020, methodology): “Recent methods such as GA-Net [59] and HSM [53] are obtaining slightly better metrics, although they require 1.”
Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, background): “Different designs of the feature net and the matching net form variants of the volumetric networks [4, 10, 11, 12, 6].”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Zhang et al. (2019b) reported that some sequences in this dataset do not provide camera poses.”
Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance (2020, background): “Nowadays, many benchmarks [40,56] in depth estimation are dominated by algorithms based on neural networks [14,66].”
Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “GANet [24] also tries to replace 3D convolutions with two guided aggregation layers, which achieves a higher accuracy using spatially dependent 3D aggregation, but at the cost of a higher aggregation time due to the two guided aggregation layers.”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “From Table 5, we can distinguish three types of methods; slow methods, e.g., PSMNet [64], DeepPruner (Best) and (Fast) [83], and GANet [85], require more than 1 second to estimate one disparity map.”
Virtual KITTI 2 (2020, methodology): “In order to show these new capabilities, we ran GANet [25] on Virtual KITTI 2 as well as on the real KITTI sequences (KITTI tracking benchmark [5]) using the provided pre-trained models.”
The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth (2021, background): “Recent approaches improved the underlying architectures and contributed more effective ways of regularizing the cost volume [7, 94, 95, 10].”