Implicit Surface Representations As Layers in Neural Networks

By Mateusz Michalkiewicz, Jhony Kaesemodel Pontes, Dominic Jack, Mahsa Baktash, Anders P. Eriksson. 2019

302 citations recorded by the source corpus.

Implicit shape representations, such as Level Sets, provide a very elegant formulation for performing computations involving curves and surfaces. However, including implicit representations into canonical Neural Network formulations is far from straightforward. This has consequently restricted existing approaches to shape inference, to significantly less effective representations, perhaps most commonly voxels occupancy maps or sparse point clouds. To overcome this limitation we propose a novel formulation that permits the use of implicit representations of curves and surfaces, of arbitrary topology, as individual layers in Neural Network architectures with end-to-end trainability. Specifically, we propose to represent the output as an oriented level set of a continuous and discretised embedding function. We investigate the benefits of our approach on the task of 3D shape prediction from a single image; and demonstrate its ability to produce a more accurate reconstruction compared to voxel-based representations. We further show that our model is flexible and can be applied to a variety of shape inference problems.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Model Architectures, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Model Architectures

In 3D Vision & Pose

Citations in the corpus

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020, background): “…volume density [27], occupancy [24], and signed distance [32], and have achieved state-of-the-art results across a variety of tasks such as shape representation [9, 10, 12, 13, 17, 26, 32], texture synthesis [15, 31], shape inference from images [22, 23], and novel view synthesis [27, 29, 35, 37].”
Dynamic View Synthesis from Dynamic Monocular Video (2021, background): “, points in 3D space) to the desired quantities such as occupancy value [30, 45, 40], signed distance [38, 2, 31], volume density [32], color [36, 48, 45, 32].”
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction (2021, methodology): “This representation has been applied successfully to shape representation [26, 27, 31, 4, 1, 10, 47, 32], novel view synthesis [38, 23, 15, 28, 21, 33, 34, 40, 37] and multi-view 3D reconstruction [46, 29, 17, 12, 22].”
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis (2020, background): “These can be supervised with 3D data, such as point clouds, and optimized as either signed distance functions [46, 38, 1, 16, 57, 20, 47, 55, 27] or occupancy networks [37, 6].”
Learning Continuous Image Representation with Local Implicit Image Function (2020, methodology): “This idea has been widely applied in modeling 3D object shapes [6, 28, 2, 13], 3D surfaces of the scene [42, 18, 36, 4] as well as the appearance of the 3D structure [33, 32, 29].”
Neural Sparse Voxel Fields (2020, background): “Most works require 3D supervision during training to infer the SDF value or the occupancy probability of any 3D point (Michalkiewicz et al., 2019; Mescheder et al., 2019; Chen and Zhang, 2019; Park et al., 2019; Peng et al., 2020), while other works learn 3D representations only from images with…”
HyperCube: Implicit Field Representations of Voxelized 3D Models (2021, unknown): “In such a case the implicit occupancy [1, 26, 27], distance ﬁeld [28, 29] and surface parametrization [30, 31, 32, 33] models use a neural network to represent a 3D object.”
Implicit Neural Representations with Periodic Activation Functions (2020, background): “Recent work has demonstrated the potential of fully connected networks as continuous, memory-efficient implicit representations for shape parts [6, 7], objects [1, 4, 8, 9], or scenes [10–12].”
Efficient Geometry-aware 3D Generative Adversarial Networks (2021, unknown): “Emerging neural scene representations use differentiable 3D-aware representations [1, 3, 6, 8, 13, 17, 43, 44, 52, 65] that can be optimized using 2D multi-view images via neural rendering [15, 20, 24, 30, 34–37, 40, 45, 46, 50, 51, 54, 62, 63, 70– 72].”
Space-time Neural Irradiance Fields for Free-Viewpoint Video (2020, methodology): “The core idea is to use a multilayer perceptron (MLPs) to implicitly model the occupancy [41, 42], signed distance functions [53, 2], object appearance [51], volumetric density [16, 36, 63, 67, 64, 44] in the 3D space.”
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision (2019, methodology): “the representation they use as voxel-based [8, 13, 19, 61, 64, 73, 82, 83], pointbased [2, 17, 31, 40, 77, 85], mesh-based [24, 32, 41, 55, 80], or implicit representations [3, 12, 22, 30, 48, 49, 56, 66, 81].”
StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation (2021, background): “Neural implicit representations (NIR) have been widely adopted for learning shape and appearance of objects [4,10, 14, 20, 45, 50, 52, 56, 57], local parts [17, 18], and full 3D scenes [7, 11, 29, 55] from explicit 3D supervisions.”
RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs (2021, background): “Neural Representations: In 3D vision, coordinatebased neural representations [7, 34, 35, 44] have become a popular representation for various tasks such as 3D reconstruction [1, 7, 13, 14, 34, 40, 43–45, 48, 51, 55, 57], 3Daware generative modelling [5,9,15,16,33,38,39,42,49,64], and novel-view synt”
Convolutional Occupancy Networks (2020, background): “Implicit Representations: More recent implicit occupancy [3,26] and distance field [27, 31] models use a neural network to infer an occupancy probability or distance value given any 3D point as input.”
AutoInt: Automatic Integration for Fast Neural Volume Rendering (2020, background): “Such representations have been explored for modeling shape parts [16, 17], objects [3, 6, 10, 19, 25, 27, 30, 34, 35, 40, 42, 43, 49, 53, 60], or scenes [14, 21, 29, 37, 45, 51].”
MetaSDF: Meta-learning Signed Distance Functions (2020, background): “Parameterized as multilayer perceptrons, these continuous representations have applications in modeling shape parts [10, 11], objects [1, 12–14], or scenes [8, 15, 16].”
Volume Rendering of Neural Implicit Surfaces (2021, background): “Recent studies have been focusing on model implicit functions with multi-layer perceptron (MLP) due to its expressive representation power and low memory foot-print, including scene (geometry & appearance) representation [9, 20, 19, 23, 25, 29, 36, 28, 35] and free-view rendering [33, 16, 30, 26, 17”