Semi-Supervised Deep Learning for Monocular Depth Map Prediction

By Yevhen Kuznietsov, Jörg Stückler, Bastian Leibe. Computer Vision and Pattern Recognition, 2017

693 citations recorded by the source corpus.

Supervised deep learning often suffers from the lack of sufficient training data. Specifically in the context of monocular depth map prediction, it is barely possible to determine dense ground truth depth images in realistic dynamic outdoor environments. When using LiDAR sensors, for instance, noise is present in the distance measurements, the calibration between sensors cannot be perfect, and the measurements are typically much sparser than the camera images. In this paper, we propose a novel approach to depth map prediction from monocular images that learns in a semi-supervised way. While we use sparse ground-truth depth for supervised learning, we also enforce our deep network to produce photoconsistent dense depth maps in a stereo setup using a direct image alignment loss. In experiments we demonstrate superior performance in depth map prediction from single images compared to the state-of-the-art methods.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6088a5af7198e763f9d6a9a5e78f45302228a3ff

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Depth with Convolutional Spatial Propagation Network (2018, methodology): “Depending on DCN, numerous algorithms are developed, including supervised methods [12], [13], [35], [36], [37], [38], [39] semi-supervised methods [40] or unsupervised methods [41], [42], [43], [44], [45], [46].”
Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction (2018, methodology): “Following [6] methods like [9][19][41] use the photometric error based supervision to learn depth estimators”
From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation (2019, methodology): “While most of the state-of-the-art works apply models based on deep convolutional neural netowrks (DCNNs) in supervised fashion, some works proposed semi- [24] or selfsupervised learning methods which do not fully rely on the ground truth depth data.”
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network (2018, methodology): “Numerous algorithms are developed through supervised methods [16,3,4,17], semi-supervised methods [18] or unsupervised methods [19,20,21,22].”
SegStereo: Exploiting Semantic Information for Disparity Estimation (2018, methodology): “[27] where supervised and unsupervised alignment loss are used to train the network for depth estimation.”
Real-Time Monocular Depth Estimation Using Synthetic Data with Domain Adaptation via Image Style Transfer (2018, methodology): “The training in the work of [38] is supervised by sparse ground truth depth and the model is then enforced within a stereo framework via an image alignment loss to output dense depth.”
Monocular depth estimation based on deep learning: An overview (2020, methodology): “ire. Therefore, in this section, we review the monocular depth estimation methods from the aspect of using ground truth: supervised methods [49], unsupervised methods [50] and semi-supervised methods [48]. Although the training processes of the unsupervised and semi-supervised methods rely on monocu”
FastDepth: Fast Monocular Depth Estimation on Embedded Systems (2019, background): “Semi-supervised [19] and unsupervised learning [20–22] setups have also been explored for disparity image prediction.”
Deep Ordinal Regression Network for Monocular Depth Estimation (2018, methodology): “To improve efﬁciency, Roy and Todorovic [48] proposed the Neural Regression Forest method which allows for parallelizable training of “shallow” CNNs. Recently, unsupervised or semi-supervised learning is introduced to learn depth estimation networks [17, 33].”
Visual SLAM for Automated Driving: Exploring the Applications of Deep Learning (2018, background): “In table 2 we summarize three Table 2: Raw Depth competition on KITTI [27] from [37].”
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations (2018, methodology): “using supervised learning [9], [26]–[28], unsupervised learning [29], [30] and semi-supervised learning [31].”
SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation (2018, background): “Recent advances in Convolutional Neural Networks (CNNs), especially for depth and pose estimation [1], [2], [3], [4] from a monocular camera have dramatically shifted the landscape of single-image 3D reconstruction.”
Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation (2019, background): “Recently, several works proposed to tackle the depth estimation problem within an unsupervised learning framework [20, 28, 34, 42, 32].”
BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation (2022, methodology): “To combine the advantages of both training strategies, semi-supervised based MDE methods [21], [22]propose to simultaneously utilize supervised loss and self-supervised loss terms during the training.”
DeepV2D: Video to Depth with Differentiable Structure from Motion (2018, background): “Kuznietzov et al. [22] combines photo-consistency with velodyne data.”
Vision by alignment (2018, methodology): “[2015] and the depth estimation network of Kuznietsov et al. [2017] both have similar topology to the network structure depicted in Figure 37, namely, a directed acyclic graph (DAG) consisting of a main backbone with skip connections.”