DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

By Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomír Mech, Ulrich Neumann. 2019

619 citations recorded by the source corpus.

Reconstructing 3D shapes from single-view images has been a long-standing research problem. In this paper, we present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image by predicting the underlying signed distance fields. In addition to utilizing global image features, DISN predicts the projected location for each 3D point on the 2D image, and extracts local features from the image feature maps. Combining global and local features significantly improves the accuracy of the signed distance field prediction, especially for the detail-rich areas. To the best of our knowledge, DISN is the first method that constantly captures details such as holes and thin structures present in 3D shapes from single-view images. DISN achieves the state-of-the-art single-view reconstruction performance on a variety of shape categories reconstructed from both synthetic and real images. Code is available at this https URL The supplementary can be found at this https URL

Subfields: Computer Vision and Pattern Recognition, Geology, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Geology

In 3D Vision & Pose

Citations in the corpus

Multiview Compressive Coding for 3D Reconstruction (2023, background): “One line of work trains models that predict 3D geometry via CAD [16, 68], meshes [30,75], voxels [15,72] or point clouds [12,36] supervision.”
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization (2023, background): “Recently, there has been an increasing number of work on learning to generate a 3D implicit ﬁeld from a single image [84, 45, 66, 24, 53, 17, 20, 26, 50, 78, 49].”
pixelNeRF: Neural Radiance Fields from One or Few Images (2020, methodology): “Most existing methods [49, 25, 28, 39] predict in canonical space, where all objects of a semantic category are aligned to a consistent orientation.”
Neural Scene Graphs for Dynamic Scenes (2020, methodology): “Existing methods have proposed to learn features on discrete geometric primitives, such as points [2, 27], multi-planes [10, 17, 21, 35, 46], meshes [6, 39] higherorder primitives like voxel grids [16, 31, 44], or implicitly represent features [15, 20, 24, 26, 43, 22] using functions F : R(3) → R th”
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization (2019, background): “Note that recently [18] extend the idea of PIFu by explicitly combining global features and local features, demonstrating globally coherent and locally detailed reconstruction for general objects is possible.”
D-NeRF: Neural Radiance Fields for Dynamic Scenes (2020, background): “The first works exploiting implicit representations [24, 31, 6, 47] for 3D representation were limited by their requirement of having access to 3D ground-truth geometry, often expensive or even impossible to obtain for in the wild scenes.”
Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers (2023, positive): “Previous works have constructed priors based on collections of 3D primitives using an encoder-decoder approach [42, 75].”
LRM: Large Reconstruction Model for Single Image to 3D (2023, neutral): “…previous works apply an encoder-decoder framework to model the image-to-3D data distribution (Choy et al., 2016; Yan et al., 2016; Dai et al., 2017; Xu et al., 2019; Wu et al., 2020; M¨uller et al., 2022; Sajjadi et al., 2022; Goel et al., 2023), where a compact latent code is trained to carry the…”
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction (2020, methodology): “This motivates later works of PIFu [21] and DISN [27].”
Shape As Points: A Differentiable Poisson Solver (2021, methodology): “Neural implicit representations have been successfully used to represent geometries of objects [10,16,18,37,42,46,49,52,60,62,65,66] and scenes [8,28,36,48,54,60].”
IBRNet: Learning Multi-View Image-Based Rendering (2021, background): “Earlier work [2, 14, 23, 36, 41, 53, 61] showed that MLPs can be used as implicit shape representations, where the weights of the MLPs map continuous spatial coordinates to signed distance or occupancy values.”
Learning Continuous Image Representation with Local Implicit Image Function (2020, methodology): “Instead of using auto-decoder, our work adopts the auto-encoder architecture [27, 6, 38, 39, 47], which gives an efficient and effective manner for sharing knowledge between a large variety of samples.”
GRNet: Gridding Residual Network for Dense Point Cloud Completion (2020, background): “Therefore the networks tend to output a mean shape that minimizes the distance, which causes the loss of the object’s details [19,20].”
BSP-Net: Generating Compact Meshes via Binary Space Partitioning (2019, background): “Up to now, deep neural networks for shape analysis and synthesis have been developed mainly for voxel grids [14, 18, 48, 50], point clouds [1, 33, 34, 55, 56], and implicit functions [5, 13, 22, 28, 52].”
Zero-1-to-3: Zero-shot One Image to 3D Object (2023, background): “One line of work builds priors from relying on collections of 3D primitives represented as meshes [58, 62], voxels [16, 60], or point clouds [12, 30], and use image encoders for conditioning.”
Dynamic View Synthesis from Dynamic Monocular Video (2021, background): “Continuous and differentiable functions parameterized by fully-connected networks (also known as multilayer perceptron, or MLPs) have been successfully applied as compact, implicit representations for modeling 3D shapes [10, 59, 42, 18, 17], object appearances [40, 37], 3D scenes [52, 35, 44].”