Hierarchical Neural Architecture Search for Deep Stereo Matching

By Xuelian Cheng, Yiran Zhong, Mehrtash Harandi, Yuchao Dai, Xiaojun Chang, Tom Drummond, Hongdong Li, ZongYuan Ge. 2020

429 citations recorded by the source corpus.

To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation. The underlying idea for the NAS algorithm is straightforward, namely, to enable the network the ability to choose among a set of operations (e.g., convolution with different filter sizes), one is able to find an optimal architecture that is better adapted to the problem at hand. However, so far the success of NAS has not been enjoyed by low-level geometric vision tasks such as stereo matching. This is partly due to the fact that state-of-the-art deep stereo matching networks, designed by humans, are already sheer in size. Directly applying the NAS to such massive structures is computationally prohibitive based on the currently available mainstream computing resources. In this paper, we propose the first end-to-end hierarchical NAS framework for deep stereo matching by incorporating task-specific human knowledge into the neural architecture search framework. Specifically, following the gold standard pipeline for deep stereo matching (i.e., feature extraction -- feature volume construction and dense matching), we optimize the architectures of the entire pipeline jointly. Extensive experiments show that our searched network outperforms all state-of-the-art deep stereo matching architectures and is ranked at the top 1 accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the top 1 on SceneFlow dataset with a substantial improvement on the size of the network and the speed of inference. The code is available at this https URL.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Comprehensive Survey of Neural Architecture Search (2020, background): “NAS aims to design a neural architecture that achieves the best possible performance using limited computing resources in an automated way with minimal human intervention [26, 114].”
Efficient Attention: Attention with Linear Complexities (2018, neutral): “Ta-Table Model mAP@0.1 mAP@0.2 mAP@0.3 mAP@0.4 mAP@0.5 mAP@0.6 mAP@0.7 1.09 CSPN [5] 0.78 LEAStereo [6] 0.78 EA-PSMNet 0.48”
Attention Concatenation Volume for Accurate and Efficient Stereo Matching (2022, methodology): “LEAStereo [3] has poor accuracy on Scene Flow; the performance of HITNet [18] in real-world scenarios (KITTI and ETH3D) is far inferior to our ACVNet.”
One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting (2020, background): “N EURAL architecture search (NAS) has recently attracted massive interest from the deep learning community because experts do not have inordinate amounts of time and labor designing neural networks [13], [18], [29], [35], [42], [48], [51], [72].”
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers (2020, methodology): “Following the same idea of computing matching cost by learnt 3D convolutions, other works [40, 7, 41, 15] attempt to enable high resolution inference, mitigate the memory constraint, and/or leverage richer context information via a multi-resolution approach.”
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search (2021, methodology): “To avoid repeated training of candidate networks, weight-sharing NAS methods [7, 43, 18, 1, 6, 13, 82] optimize a supernet that encodes the whole search space, then rate each candidate architecture according to its weights inherited from the supernet.”
Dynamic Slimmable Network (2021, neutral): “In recent years, many different approaches have been proposed to improve the inference efficiency of neural networks, including network pruning [40, 47, 19, 20, 48, 50], weight quantization [30], knowledge distillation [2, 52, 21], manually and automatically designing of efficient networks [56, 53, ”
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation (2022, methodology): “We achieve competitive performance on both datasets , surpassing LEAStereo [8] in KITTI 2012 by 9.”
Iterative Geometry Encoding Volume for Stereo Matching (2023, methodology): “On KITTI 2012, our IGEVStereo outperforms LEAStereo [9] and RAFT-Stereo [24] by 10.”
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching (2020, neutral): “Here we show that 0.74 0.32 LEAStereo [8] 0.78 0.3 GA-Net [61] 0.84 1.6 PSMNet [7] 1.09 0.41 StereoNet [25] 1.1 0.015 our architecture is more effective.”
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (2021, background): “…design, with achieving remarkable success on image recognition (Tan & Le, 2019; Li et al., 2021b; 2020), language modeling (Jiang et al., 2019), and other deep learning ap-1 plications (Ren et al., 2020; Cheng et al., 2020; Chen et al., 2019b; Hu et al., 2021; Zhu et al., 2021; Ren et al., 2021).”
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching (2021, background): “Method KITTI2012 3px(%) KITTI2015 D1 all(%) time (s) Noc All Noc All LEAStereo [6] 1.”