FastDepth: Fast Monocular Depth Estimation on Embedded Systems

By Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, Vivienne Sze. IEEE International Conference on Robotics and Automation, 2019

373 citations recorded by the source corpus.

Depth sensing is a critical function for robotic tasks such as localization, mapping and obstacle detection. There has been a significant and growing interest in depth estimation from a single RGB image, due to the relatively low cost and size of monocular cameras. However, state-of-the-art single-view depth estimation algorithms are based on fairly complex deep neural networks that are too slow for real-time inference on an embedded platform, for instance, mounted on a micro aerial vehicle. In this paper, we address the problem of fast depth estimation on embedded systems. We propose an efficient and lightweight encoder-decoder network architecture and apply network pruning to further reduce computational complexity and latency. In particular, we focus on the design of a low-latency decoder. Our methodology demonstrates that it is possible to achieve similar accuracy as prior work on depth estimation, but at inference speeds that are an order of magnitude faster. Our proposed network, FastDepth, runs at 178 fps on an NVIDIA Jetson TX2 GPU and at 27 fps when using only the TX2 CPU, with active power consumption under 10 W. FastDepth achieves close to state-of-the-art accuracy on the NYU Depth v2 dataset. To the best of the authors’ knowledge, this paper demonstrates real-time monocular depth estimation using a deep neural network with the lowest latency and highest throughput on an embedded platform that can be carried by a micro aerial vehicle.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/891d2e08bef676542a3979aa4083594643730b4a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation (2020, background): “(Wofk et al. 2019) proposed a lightweight supervised depth estimation network, who use encoder-decoder architecture and include 1.”
NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization (2021, methodology): “The comparison between the proposed NetAdaptV2 and NetAdaptV1 [1], which is used in FastDepth [42], is summarized in Table 7.”
Depth Anything V2 (2024, neutral): “…screens) [15], etc. • contain fine details (comparable to the details of Marigold) in the predicted depth maps, including but not limited to thin objects ( e . g ., chair legs) [42], small holes, etc. • provide varied model scales and inference efficiency to support extensive applications [82].”
Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation (2019, background): “More recent works study how to obtain 3D data from color images by directly inferring depth images [14, 29, 10, 40, 22, 53, 16, 45], or filling in missing depth values in RGB-D images captured by commodity 3D cameras [21, 17, 63].”
Learning to Upsample by Learning to Sample (2023, background): “Dense prediction refers to a branch of tasks that require point-wise label prediction, such as semantic/instance/panoptic segmentation [2, 39, 40, 8, 7, 13, 11, 16, 19], object detection [33, 4, 24, 36], and monocular depth estimation [38, 18, 3, 21].”
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, background): “Monocular Depth Estimation (MDE), which is a fundamental problem with broad applications in robotics [65], autonomous driving [63, 79], virtual reality [47], etc ., also requires a foundation model to estimate depth information from a single image.”
Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation (2019, methodology): “We use the pre-trained network from [36] with no modifications.”
Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance? (2019, positive): “Thus, inspired by the winning entry in the RGB track of the Habitat Challenge 2019 [13], we also test an agent that uses a monocular depth estimator [37] to predict depth from RGB, which is then fed to the navigation 2 Other metrics such as rank correlation can also be used.”
MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer (2022, background): “Other works, instead, focused on the design of efficient and fast networks suitable for lowpowered embedded devices [34, 33, 55].”