DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
By J. Park, Peter R. Florence, Julian Straub, Richard A. Newcombe, Steven J. Lovegrove. 2019
4,653 citations recorded by the source corpus.
Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to representing 3D geometry for rendering and reconstruction. These provide trade-offs across fidelity, efficiency and compression capabilities. In this work, we introduce DeepSDF, a learned continuous Signed Distance Function (SDF) representation of a class of shapes that enables high quality shape representation, interpolation and completion from partial and noisy 3D input data. DeepSDF, like its classical counterpart, represents a shape's surface by a continuous volumetric field: the magnitude of a point in the field represents the distance to the surface boundary and the sign indicates whether the region is inside (-) or outside (+) of the shape, hence our representation implicitly encodes a shape's boundary as the zero-level-set of the learned function while explicitly representing the classification of space as being part of the shapes interior or not. While classical SDF's both in analytical or discretized voxel form typically represent the surface of a single shape, DeepSDF can represent an entire class of shapes. Furthermore, we show state-of-the-art performance for learned 3D shape representation and completion while reducing the model size by an order of magnitude compared with previous work.
Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Andreas Geiger
- Gerard Pons-Moll
- Michael Niemeyer
- Michael J. Black
- Jonathan T. Barron
- Christian Theobalt
- Matthias Nießner
- Songyou Peng
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Michael Niemeyer
- Pratul P. Srinivasan
- Matthias Nießner
- Matthew Tancik
- Justin Johnson
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Andrea Vedaldi
- Andrew Zisserman
In 3D Vision & Pose
- Andreas Geiger
- Michael Niemeyer
- Matthias Nießner
- Gerard Pons-Moll
- Jonathan T. Barron
- Jitendra Malik
- Justin Johnson
- Christian Theobalt
Citations in the corpus
- Multiview Compressive Coding for 3D Reconstruction (2023, methodology): “Implicit 3D representations such as SDFs [43,52] and occupancy nets (OccNets) [36] have proven effective 3D representations.”
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020, background): “…to represent images [28, 38] (referred to as “compositional pattern producing networks”), volume density [27], occupancy [24], and signed distance [32], and have achieved state-of-the-art results across a variety of tasks such as shape representation [9, 10, 12, 13, 17, 26, 32], texture synthesis…”
- Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “Most of these methods are 1 Also see [8,48,51] for concurrent work that proposes similar ideas. either based on variational auto-encoders [39,59] or generative adversarial networks [25].”
- Learning Implicit Fields for Generative Shape Modeling (2018, positive): “single image superresolution using deep networks, e.g., [12,27], which are trained with low- and high-resolution image pairs. Progressive training [24] is another technique to improve image 1Also see [23,33] for concurrent works that propose similar ideas. quality, and we adopt it in our work to red”
- Implicit Behavioral Cloning (2021, background): “There is also a surge of interest in geometry representation learning in implicit representations [44, 45, 46, 47].”
- Learning Gradient Fields for Shape Generation (2020, positive): “Recently, implicit representations are gaining increasing popularity, as they are capable of representing shapes with high level of detail [47,39,11,40].”
- IBRNet: Learning Multi-View Image-Based Rendering (2021, background): “Earlier work [2, 14, 23, 36, 41, 53, 61] showed that MLPs can be used as implicit shape representations, where the weights of the MLPs map continuous spatial coordinates to signed distance or occupancy values.”
- Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video (2020, methodology): “This is based on the auto-decoding framework used in DeepSDF and earlier works [80, 57].”
- Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (2023, background): “They learn a continuous function that takes as input the 3D coordinate of a point and outputs the representation of this point [34, 35, 37].”
- SAL: Sign Agnostic Learning of Shapes From Raw Data (2019, methodology): “Second, we approximate a signed distance function, h̄2, to the data Xi in two different ways, and regress them using an MLP as in DeepSDF [30]; we call these methods SignReg.”
- Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis (2021, positive): “Recently, neural implicit representations [8, 39, 42, 51], which use a neural network to implicitly represent a shape via a signed distance field (SDF) or an occupancy field (OF), have emerged as an effective 3D representation.”
- Local Deep Implicit Functions for 3D Shape (2019, methodology): “Most recently, representing shape as a vector with an implicit surface function decoder has become popular, with methods such as OccNet [24], ImNet [7], DeepSDF [27], and DISN [45].”
- Learning Shape Templates With Structured Implicit Functions (2019, methodology): “We argue that our structured implicit representation is superior for template learning compared to decoding voxels [6, 55, 56], sparse-voxel octrees [50], points [11], meshes [15, 22, 53], box primitives [51], signed-distance function estimators [40], or indicator function estimators [34].”
- Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance (2020, methodology): “rendering. Implementation details. The MLP F(x;) = (f(x;);z(x;)) 2R R256 consists of 8 layers with hidden layers of width 512, and a single skip connection from the input to the middle layer as in [42]. We initialize the weights 2Rmas in [2], so that f(x;) produces an approximate SDF of a unit spher”
- Advances in Neural Rendering (2021, result): “In comparison to the previous work on using neural networks to represent implicit surfaces [PFS ∗ 19, CZ19] or volumes [MON ∗ 19], this allows NeRF’s MLP to represent much higher frequency signals without increasing its capacity (in terms of number of network weights).”
- Efficient Geometry-aware 3D Generative Adversarial Networks (2021, background): “Emerging neural scene representations use differentiable 3D-aware representations [1, 3, 6, 8, 13, 17, 43, 44, 52, 65] that can be optimized using 2D multi-view images via neural rendering [15, 20, 24, 30, 34–37, 40, 45, 46, 50, 51, 54, 62, 63, 70– 72].”