Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation

By Eddy Ilg, Tonmoy Saikia, Margret Keuper, Thomas Brox. 2018

224 citations recorded by the source corpus.

Occlusions play an important role in disparity and optical flow estimation, since matching costs are not available in occluded areas and occlusions indicate depth or motion boundaries. Moreover, occlusions are relevant for motion segmentation and scene flow estimation. In this paper, we present an efficient learning-based approach to estimate occlusion areas jointly with disparities or optical flow. The estimated occlusions and motion boundaries clearly improve over the state-of-the-art. Moreover, we present networks with state-of-the-art performance on the popular KITTI benchmark and good generic performance. Making use of the estimated occlusions, we also show improved results on motion segmentation and scene flow estimation.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, methodology): “7: DispNet-CSS[2] and our DispNormNet.”
AutoDispNet: Improving Disparity Estimation With AutoML (2019, methodology): “We compare the optimized network to the already welltweaked version of DispNet [37] to investigate who is more successful in tweaking: the student or the numerical optimization procedure.”
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (2019, methodology): “We compare our upsampling layer with the refinement network from [26, 27], adding it to our network based on a PWC-Net backbone with Bi, Occ, IRR, and the bilateral refinement layer enabled.”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “Disparity estimation networks, such as [22], [68], iResNet [63], DispNet3 [75], EdgeStereo [76], HD 3 [80], and [83], [84], use 1D correlations.”
Hierarchical Deep Stereo Matching on High-Resolution Images (2019, background): “Recent “deep” stereo methods perform well on low-resolution benchmarks [5, 11, 16, 21, 38], while failing to produce SOTA results on high-res benchmarks [26].”
Synthetic Data for Deep Learning (2019, methodology): “The Flying Chairs dataset was also later extended with additional modalities to ChairsSDHom [271] with optical flow ground trugh and Flying Chairs 2 [272] with occlusion weights and motion boundaries.”