Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation

By Andrea Pilzer, Stéphane Lathuilière, Nicu Sebe, Elisa Ricci. 2019

140 citations recorded by the source corpus.

Nowadays, the majority of state of the art monocular depth estimation techniques are based on supervised deep learning models. However, collecting RGB images with associated depth maps is a very time consuming procedure. Therefore, recent works have proposed deep architectures for addressing the monocular depth prediction task as a reconstruction problem, thus avoiding the need of collecting ground-truth depth. Following these works, we propose a novel self-supervised deep model for estimating depth maps. Our framework exploits two main strategies: refinement via cycle-inconsistency and distillation. Specifically, first a student network is trained to predict a disparity map such as to recover from a frame in a camera view the associated image in the opposite view. Then, a backward cycle network is applied to the generated image to re-synthesize back the input image, estimating the opposite disparity. A third network exploits the inconsistency between the original and the reconstructed input frame in order to output a refined depth map. Finally, knowledge distillation is exploited, such as to transfer information from the refinement network to the student. Our extensive experimental evaluation demonstrate the effectiveness of the proposed framework which outperforms state of the art unsupervised methods on the KITTI benchmark.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Knowledge Distillation: A Survey (2020, background): “2019), depth estimation (Pilzer et al. 2019; Ye et al. 2019), visual odometry (Saputra et al.”
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion (2019, methodology): “These methods and improved variants [1, 53, 57, 67, 73] enable teaching students with goals such as quantization [42, 55], compact neural network architecture design [57], semantic segmentation [31], self-distillation [14], and un-/semi-supervised learning [34, 54, 70].”
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “[181] propose an unsupervised distillation approach where the left image is translated to right via image translation framework [97], [222], and the inconsistencies between left and right images are used to improve depth estimation, which is finally used to improve the student network via KD.”
Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance (2020, methodology): “Other approaches employ teacher-student learning [45], Generative Adversarial Networks (GANs) [2,10,46], proxy labels from traditional stereo algorithms [54], or recurrent neural networks [59,67].”
On the Uncertainty of Self-Supervised Monocular Depth Estimation (2020, methodology): “Other methods improved efficiency [53, 50] to enable deployment on embedded devices, or accuracy by simulating a trinocular setup [56], jointly learning for semantic [79], using higher resolution [51], GANs [1], sparse inputs from visual odometry [2] or a teacher-student scheme [52].”
Monocular depth estimation based on deep learning: An overview (2020, background): “Methods based on adversarial learning and knowledge distillation: Combining advanced network frameworks, like adversarial learning [31], [102], [65] and knowledge distillation [103], is becoming popular and can significantly improve the performance.”