BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation

By Zhenyu Li, Xuyang Wang, Xianming Liu, Junjun Jiang. IEEE Transactions on Image Processing, 2022

295 citations recorded by the source corpus.

Monocular depth estimation (MDE) is a fundamental task in computer vision and has drawn increasing attention. Recently, some methods reformulate it as a classification-regression task to boost the model performance, where continuous depth is estimated via a linear combination of predicted probability distributions and discrete bins. In this paper, we present a novel framework called BinsFormer, tailored for the classification-regression-based depth estimation. It mainly focuses on two crucial components in the specific task: 1) proper generation of adaptive bins; and 2) sufficient interaction between probability distribution and bins predictions. To specify, we employ a Transformer decoder to generate bins, novelly viewing it as a direct set-to-set prediction problem. We further integrate a multi-scale decoder structure to achieve a comprehensive understanding of spatial geometry information and estimate depth maps in a coarse-to-fine manner. Moreover, an extra scene understanding query is proposed to improve the estimation accuracy, which turns out that models can implicitly learn useful information from the auxiliary environment classification task. Extensive experiments on the KITTI, NYU, and SUN RGB-D datasets demonstrate that BinsFormer surpasses state-of-the-art MDE methods with prominent margins. Code and pretrained models are made publicly available at https://github.com/zhyever/ Monocular-Depth-Estimation-Toolbox/tree/main/configs/ binsformer.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

View Paper: https://www.semanticscholar.org/paper/2db0c931ee1f24ec2b76459a28974763c9da253f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (2024, unknown): “A line of work sought to improve metric depth prediction through a global distribution of depth values (Fu et al., 2018; Bhat et al., 2021; 2022; Li et al., 2024b) and further conditioning on scene type (Bhat et al., 2023).”
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, background): “Following this, many works consistently improve the depth estimation accuracy by carefully designing the regression task as a classification task [3, 34], introducing more priors [32, 53, 75, 82], and better objective functions [67, 77], etc .”
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth (2023, methodology): “The metric bins module outputs metric depth and follows the adaptive binning principle, originally introduced in [5] and subsequently modiﬁed by [2,6,26,36].”
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes (2022, result): “Our results still lags behind the most recent state-of-the-art model for this task [29] which consists of a mixed classiﬁcation/regression loss, adaptive bins, auxiliary scene classiﬁcation task and multi-scale prediction reﬁning.”
DINOv2: Learning Robust Visual Features without Supervision (2023, methodology): “First, we run our evaluations for MAE (He et al., 2022), DINO (Caron et al., 2021), SEERv2 (Goyal et al., 2022a), MSN (Assran et al., 2022), EsViT (Li et al., 2022a), Mugs (Zhou et al., 2022b) and iBOT (Zhou et al., 2022a).”
Images Speak in Images: A Generalist Painter for In-Context Visual Learning (2022, methodology): “ADE20K [54] is a widely-used semantic segmentation dataset, covering a broad range of 150 semantic categories.”
MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision (2024, unknown): “A number of methods [3, 4, 16, 24, 33, 38, 66] have been developed to estimate depth with metric scale, which often rely heavily on data from specific sensors such as RGBD cameras, LiDAR, and calibrated stereo cameras.”
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks (2022, methodology): “Notably, we achieve this without the extensive data augmentations methods typically used by SOTA models (Yu et al., 2022a; He et al., 2022).”
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (2023, unknown): “Some authors treat depth estimation as a combined regression-classification task, using various binning strategies like Ad-aBins [4] or BinsFormer [26] to discretize depth range.”