AANet: Adaptive Aggregation Network for Efficient Stereo Matching

By Haofei Xu, Juyong Zhang. 2020

602 citations recorded by the source corpus.

Despite the remarkable progress made by learning based stereo matching algorithms, one key challenge remains unsolved. Current state-of-the-art stereo models are mostly based on costly 3D convolutions, the cubic computational complexity and high memory consumption make it quite expensive to deploy in real-world applications. In this paper, we aim at completely replacing the commonly used 3D convolutions to achieve fast inference speed while maintaining comparable accuracy. To this end, we first propose a sparse points based intra-scale cost aggregation method to alleviate the well-known edge-fattening issue at disparity discontinuities. Further, we approximate traditional cross-scale cost aggregation algorithm with neural network layers to handle large textureless regions. Both modules are simple, lightweight, and complementary, leading to an effective and efficient architecture for cost aggregation. With these two modules, we can not only significantly speed up existing top-performing models (e.g., 41x than GC-Net, 4x than PSMNet and 38x than GA-Net), but also improve the performance of fast stereo models (e.g., StereoNet). We also achieve competitive results on Scene Flow and KITTI datasets while running at 62ms, demonstrating the versatility and high efficiency of the proposed method. Our full framework is available at https://github.com/haofeixu/aanet.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, methodology): “In Table 3, we report the latest benchmark of volumetric based stereo matching method [11] and direct regression approaches [5, 26] for comparisons.”
GMFlow: Learning Optical Flow via Global Matching (2021, methodology): “To tackle this issue, we use a differentiable matching layer [19,43,47].”
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation (2022, methodology): “One line of networks [12, 21, 22, 27, 29, 44, 49] only uses 2D convolutions.”
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (2021, methodology): “Many previous work, including Dispnet [18], MADNet [29], and AANet [32], belong to this category.”
PatchmatchNet: Learned Multi-View Patchmatch Stereo (2020, methodology): “To prevent the problem of aggregating across surface boundaries, we propose an adaptive spatial aggregation strategy based on Patchmatch stereo [3] and AANet [37].”
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers (2020, methodology): “Under multiple evaluation settings, we compare STTR with prior work spanning the primary learning-based stereo depth paradigms, including correlation-based AANet [39], 3D convolution-based PSMNet [5] and GANet-11 [43], correlation and 3D convolution hybrid approach GwcNet-g [16] and a classification”
Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “To reduce the complexity, AANet [20] proposes an intra-scale and cross-scale cost aggregation algorithm to replace the conventional 3D convolutions which can achieve very fast inference speed with a sacrifice of nontrivial accuracy degradation.”
Unifying Flow, Stereo and Depth Estimation (2022, methodology): “[19], [20], [21], [22] have been proposed to handle different tasks, ignoring the fact that many multi-view geometry and motion tasks are fundamentally related correspondence estimation problems.”
Unknown Object Segmentation from Stereo Images (2021, methodology): “In addition, we experiment with utilizing the predicted depth of both [71] as well as INSTR as input for the depth-only version of [4], and receive 33.42 and 17.97 % mIoU on rc visard scenes of STIOS.”
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images (2024, neutral): “Next, we discuss the key components.”