Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

By Xian-Feng Han, Hamid Laga, Mohammed Bennamoun. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

417 citations recorded by the source corpus.

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4032bdd102c7b815b4ce719a471e09aee7411e7c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning for 3D Point Clouds: A Survey (2019, positive): “These datasets have further boosted the research of deep learning on 3D point clouds, with an increasingly number of methods being proposed to address various problems related to point cloud processing, including 3D shape classification, 3D object detection and tracking, 3D point cloud segmentation,”
Multi-View Mesh Reconstruction with Neural Deferred Shading (2022, neutral): “Traditionally, it has been approached by matching pixels between images, often based on photo-consistency constraints or learned features [16, 25].”
A Survey on 3D Gaussian Splatting (2024, neutral): “Please see [74, 115, 245, 256, 273] for a comprehensive overview of radiance fields, scene reconstruction and representation, and rendering methods.”
Artificial intelligence in the creative industries: a review (2020, background): “A review of state-of-the-art 3D reconstruction from images using deep learning can be found in [291].”
VoxFormer: Sparse Voxel Transformer for Camera-Based 3D Semantic Scene Completion (2023, background): “This challenging problem receives extensive attention in the traditional computer vision era [10] and the recent deep learning era [11].”
MonoScene: Monocular 3D Semantic Scene Completion (2021, background): “A comprehensive survey on the matter is [30].”
A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation (2020, methodology): “Many papers attempted to address this problem for monocular depth estimation and for 3D shape reconstruction by jointly optimising for the camera parameters and the geometry of the 3D scene [153].”
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users (2020, methodology): “In addition, modern deep learning-based 3D object reconstruction [72] and disparity estimation in stereo vision [73] rely on self-supervised learning to overcome the time-consuming manual annotation of training data.”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, background): “For a more exhaustive recent review, we refer the reader to [23].”