Real-Time Monocular Depth Estimation Using Synthetic Data with Domain Adaptation via Image Style Transfer

By Amir Atapour-Abarghouei, Toby P. Breckon. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

294 citations recorded by the source corpus.

Monocular depth estimation using learning-based approaches has become promising in recent years. However, most monocular depth estimators either need to rely on large quantities of ground truth depth data, which is extremely expensive and difficult to obtain, or predict disparity as an intermediary step using a secondary supervisory signal leading to blurring and other artefacts. Training a depth estimation model using pixel-perfect synthetic data can resolve most of these issues but introduces the problem of domain bias. This is the inability to apply a model trained on synthetic data to real-world scenarios. With advances in image style transfer and its connections with domain adaptation (Maximum Mean Discrepancy), we take advantage of style transfer and adversarial training to predict pixel perfect depth from a single real-world color image based on training over a large corpus of synthetic environment data. Experimental results indicate the efficacy of our approach compared to contemporary state-of-the-art techniques.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/21547e038b383fb9d2ca14eb9978b1d4d8e6b178

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Digging Into Self-Supervised Monocular Depth Estimation (2018, background): “in the form of known object sizes [66], sparse ordinal depths [77, 6], supervised appearance matching terms [72, 73], or unpaired synthetic depth data [45, 2, 16, 78], all while still requiring the collection of additional depth or other annotations.”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “ata points) [100], in situations with different label sets for each domain [223], for object detection [39, 97, 106], for person re-identification [11,57,73,142,251,287,288], and for depth estimation [8,155,174]. 7.2 Natural Language Processing Domain adaptation has been used in natural language pro”
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion (2021, background): “4 that DanceTrack only provides bounding box annotation, for now, there would be several interesting future works: (1) extending its annotation modalities, (2) using weaklysupervised learning [34,38,49] to estimate other modalities, (3) using transfer learning and domain adaptation [1, 5, 22] to tra”
Learning Semantic Segmentation From Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach (2018, unknown): “Besides image classification, domain adaptation has been studied in other vision tasks such as object detection [4], depth estimation [1] etc.”
On the Uncertainty of Self-Supervised Monocular Depth Estimation (2020, background): “In this case, labels can be sourced from synthetic datasets [46], used to train stereo networks for single view stereo [42] and label distillation [22] or in alternative to learn depth estimation and perform domain transfer when dealing with real images [4].”
Domain Adaptation for Image Dehazing (2020, background): “Domain adaptation aims to reduce the discrepancy between different domains [1, 6, 20].”
Consistent video depth estimation (2020, background): “pth-Estimation/ depth maps that are difficult to acquire. Several efforts have been made to address this issue, e.g., training on synthetic dataset [Mayer et al. 2016a] followed by domain adaptation [Atapour-Abarghouei and Breckon 2018], collecting relative depth annotations [Chen et al. 2016], usin”
Neural Style Transfer: A Review (2017, methodology): “delighted to find that related researches on NST also bring new inspirations for other areas [106], [107], [108] and accelerate the development of a wider vision community.”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, background): “Zhao et al. [147] consider both synthetic-to-real [146] and real-to-synthetic [145], [148] translations.”