Self-Supervised Monocular Depth Hints

By Jamie Watson, Michael Firman, Gabriel J. Brostow, Daniyar Turmukhambetov. IEEE International Conference on Computer Vision, 2019

294 citations recorded by the source corpus.

Monocular depth estimators can be trained with various forms of self-supervision from binocular-stereo data to circumvent the need for high-quality laser-scans or other ground-truth data. The disadvantage, however, is that the photometric reprojection losses used with self-supervised learning typically have multiple local minima. These plausible-looking alternatives to ground-truth can restrict what a regression network learns, causing it to predict depth maps of limited quality. As one prominent example, depth discontinuities around thin structures are often incorrectly estimated by current state-of-the-art methods. Here, we study the problem of ambiguous reprojections in depth-prediction from stereo-based self-supervision, and introduce Depth Hints to alleviate their effects. Depth Hints are complementary depth-suggestions obtained from simple off-the-shelf stereo algorithms. These hints enhance an existing photometric loss function, and are used to guide a network to learn better weights. They require no additional data, and are assumed to be right only sometimes. We show that using our Depth Hints gives a substantial boost when training several leading self-supervised-from-stereo models, not just our own. Further, combined with other good practices, we produce state-of-the-art depth predictions on the KITTI benchmark.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0ea5cb2e27e1cc25912543798c694f812e598900

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Stefano Mattoccia
2. Fabio Tosi
3. Guan Huang
4. Chaoqiang Zhao
5. Zheng Zhu
6. Yang Tang
7. Youming Zhang
8. Xianda Guo

Citations in the corpus

• Consistent video depth estimation (2020, unknown): “Other notable efforts include using stereo information [Guo et al. 2018; Watson et al. 2019], better network architecture and training loss design [Gordon et al. 2019; Guizilini et al. 2019], scale-consistent ego-motion network [Bian et al. 2019], incorporating 3D geometric constraints [Mahjourian…”
• Monocular depth estimation based on deep learning: An overview (2020, unknown): “4.714 0.199 0.864 0.954 0.979 Chen et al. [98] 2019 Semi-sup 80m 0.118 0.905 5.096 0.211 0.839 0.945 0.977 Godard et al. [81] 2019 Semi-sup 80m 0.127 1.031 5.266 0.221 0.836 0.943 0.974 Watson et al. [114] 2019 Semi-sup 80m 0.112 0.857 4.807 0.203 0.862 0.952 0.978 Zhou et al. [43] 2017 Unsup 80m 0.”
• MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer (2022, background): “Although stereo pairs enable scale recovery, with further improvements achievable by leveraging noisy proxy labels [46, 54, 6], guidance from visual odometry [1] or trinoc-ular assumptions [37], unlabeled video sequences represent a more flexible alternative at the expense of learning camera poses…”
• On the Uncertainty of Self-Supervised Monocular Depth Estimation (2020, methodology): “Self-supervised [19, 82, 45, 56, 53] or weakly supervised [76, 65, 72] paradigms, leveraging on image reprojection and noisy labels respectively, have removed this issue and yield accuracy close to supervised methods [15], neglecting at all the deployment of additional depth sensors for labeling pur”