Deep Level Sets: Implicit Surface Representations for 3D Shape Inference

By Mateusz Michalkiewicz, Jhony Kaesemodel Pontes, Dominic Jack, Mahsa Baktash, Anders P. Eriksson. 2019

131 citations recorded by the source corpus.

Existing 3D surface representation approaches are unable to accurately classify pixels and their orientation lying on the boundary of an object. Thus resulting in coarse representations which usually require post-processing steps to extract 3D surface meshes. To overcome this limitation, we propose an end-to-end trainable model that directly predicts implicit surface representations of arbitrary topology by optimising a novel geometric loss function. Specifically, we propose to represent the output as an oriented level set of a continuous embedding function, and incorporate this in a deep end-to-end learning framework by introducing a variational shape inference formulation. We investigate the benefits of our approach on the task of 3D surface prediction and demonstrate its ability to produce a more accurate reconstruction compared to voxel-based representations. We further show that our model is flexible and can be applied to a variety of shape inference problems.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

In 3D Vision & Pose

Citations in the corpus

Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “Most of these methods are 1 Also see [8,48,51] for concurrent work that proposes similar ideas. either based on variational auto-encoders [39,59] or generative adversarial networks [25].”
Learning Gradient Fields for Shape Generation (2020, neutral): “…loss λ ( σ i ) ; Point cloud encoder f φ ; A neural network s θ ; Total number of training iterations T ; Point cloud X t 1: for t ← 1 to T do for σ ∈ { σ i } ki =1 do 4: x X for i ∈ t do 5: ˜ x i ← x i + N (0 , σ 2 I ) for end 7: ( σ, 8: end for 9: } φ, θ ← Adam ( , φ, θ ) for t ← 0 to T − 1 do”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, background): “Implicit Functions for rigid objects: Recently, neural networks have been used to learn a continuous implicit function representing shape [43, 48, 10, 44, 35].”
Implicit Geometric Regularization for Learning Shapes (2020, methodology): “Another implicit RGVF representation is used in (Michalkiewicz et al., 2019) for the task of image to shape prediction.”
DIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing (2019, background): “Very recently, the implicit functions, e.g ., continuous SDF and occupancy function, are exploited as 3D representations which show much promising performance in terms of high-frequency detail modeling and high resolution [32, 27, 28, 4].”
Grasping Field: Learning Implicit Representations for Human Grasps (2020, background): “Learning 3D object models using various types of representations has also been explored [9, 23, 33, 44, 50, 52, 82, 90, 91].”
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction (2020, background): “These recent approaches [14,44] make occupancy decisions based on local and global evidence, which results in more robust reconstruction of articulated and fine structures than decoding based on the X-Y-Z point coordinates and a global latent shape code [13,33,29,30].”
3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models (2023, background): “The key building block is a neural network which accepts a 3D coordinate as input, and outputs a scalar [Chen and Zhang 2019; Mescheder et al. 2019; Michalkiewicz et al. 2019; Park et al. 2019] or a vector [Chan et al. 2022; Mildenhall et al. 2020; Tang et al. 2022].”
Learning to Infer Implicit Surfaces without 3D Supervision (2019, background): “Recent works [22, 5, 6, 7, 8, 23] have shown promising results on supervised learning for 3D shape inference based on implicit representations.”
Neural Unsigned Distance Fields for Implicit Function Learning (2020, methodology): “IFL methods use either binary occupancies [49, 27, 15, 66, 21] or signed distance functions [53, 14, 50, 36] as shape representation for learning.”
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction (2020, methodology): “According to the universal approximation theorem [26], a neural network can be used to approximate any continuous function, including signed distance functions [34,40,35,10].”