Learning Implicit Fields for Generative Shape Modeling
By Zhiqin Chen, Hao Zhang. 2018
1,852 citations recorded by the source corpus.
We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes. An implicit field assigns a value to each point in 3D space, so that a shape can be extracted as an iso-surface. IM-NET is trained to perform this assignment by means of a binary classifier. Specifically, it takes a point coordinate, along with a feature vector encoding a shape, and outputs a value which indicates whether the point is outside the shape or not. By replacing conventional decoders by our implicit decoder for representation learning (via IM-AE) and shape generation (via IM-GAN), we demonstrate superior results for tasks such as generative shape modeling, interpolation, and single-view 3D reconstruction, particularly in terms of visual quality. Code and supplementary material are available at https://github.com/czq142857/implicit-decoder.
Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Michael Niemeyer
- Ben Mildenhall
- Pratul P. Srinivasan
- Matthew Tancik
- Justin Johnson
- Matthias Nießner
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Ming-Hsuan Yang
- Dacheng Tao
- Trevor Darrell
- Andrea Vedaldi
- Xiaogang Wang
- Andreas Geiger
In Generative Models
- Andreas Geiger
- Jun-Yan Zhu
- Richard Zhang
- Eli Shechtman
- Daniel Cohen-Or
- Ming-Hsuan Yang
- Alexei A. Efros
- Arash Vahdat
In 3D Vision & Pose
- Andreas Geiger
- Michael Niemeyer
- Jitendra Malik
- Gerard Pons-Moll
- Justin Johnson
- Jonathan T. Barron
- Angjoo Kanazawa
- Songyou Peng
Citations in the corpus
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020, neutral): “Coordinate-based MLPs have been used to represent images [28, 38] (referred to as “compositional pattern producing networks”), volume density [27], occupancy [24], and signed distance [32], and have achieved state-of-the-art results across a variety of tasks such as shape representation [9, 10, 12, ”
- Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “1Also see [8, 48, 51] for concurrent work that proposes similar ideas.”
- DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation (2019, methodology): “Recently, concurrently to our work, binary implicit surface representations were used by [12, 34], where they train deep networks across classes of shapes to classify 3D points as inside or outside of a shape.”
- Implicit Behavioral Cloning (2021, background): “There is also a surge of interest in 226 geometry representation learning in implicit representations [43, 44, 45, 46].”
- Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, background): “More recently, several papers start to focus on mechanisms for learning continuous Signed Distance Functions [39], [43] or continuous occupancy grids [157], which are less demanding in terms of memory requirement.”
- Efficient Geometry-aware 3D Generative Adversarial Networks (2021, neutral): “Emerging neural scene representations use differentiable 3D-aware representations [1, 3, 6, 8, 13, 17, 43, 44, 52, 65] that can be optimized using 2D multi-view images via neural rendering [15, 20, 24, 30, 34–37, 40, 45, 46, 50, 51, 54, 62, 63, 70– 72].”
- MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction (2022, methodology): “Seminal works [9, 40, 47] use a single MLP as the scene representation and show impressive object-level reconstruction quality, but they do not scale to more complicated or large-scale scenes due to the limited model capacity.”
- AutoInt: Automatic Integration for Fast Neural Volume Rendering (2020, background): “In Figure 4, we include supplemental results using real captured data from DeepVoxels [9].”
- Neural Radiance Flow for 4D View Synthesis and Video Processing (2020, positive): “Recently, neural continuous implicit fields [9, 17, 35, 41, 49, 51, 58, 61, 75] have been developed to address the discretization issues and limited resolution of classical 3D representations such as voxel grids [5, 11, 40, 56, 70, 71, 74], point clouds [13, 14, 54, 55] and meshes [18, 26, 28, 34, 6”
- PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (2020, methodology): “Furthermore, unlike implicit surface representations using a global feature vector [29, 32, 10], PIFu utilizes fully convolutional image features, retaining local details present in an input image.”
- What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction (2021, methodology): “However, most work has focused on watertight meshes [7, 35, 38, 40, 45] where there is a well-defined inside and outside.”
- Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild (2020, background): “Recent approaches train a deep network to predict the 3D shape from an image [11,17,13,19,48,45,10].”
- GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields (2020, background): “Implicit Functions: Using implicit functions to represent 3D geometry has gained popularity in learning-based 3D reconstruction [11, 12, 22, 59, 60, 65, 67, 69, 76] and has been extended to scene-level reconstruction [8, 13, 35, 72, 79].”
- Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes (2021, methodology): “Prior works [36, 31, 6, 8] have shown that neural networks can encode accurate 3D geometry without restrictions on topology or resolution by learning the SDF, which defines a surface by its zero level-set.”
- COIN: COmpression with Implicit Neural representations (2021, background): “Representing data with neural networks was originally proposed by [32] but has seen a recent surge in interest in the 3D vision community [27, 26, 6].”
- Texture Fields: Learning Texture Representations in Function Space (2019, background): “Several works [6, 17, 25, 28] proposed to formulate 3D reconstruction as learning a continuous occupancy function or a signeddistance field in 3D space, parameterized via a neural network.”
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis (2020, methodology): “Implicit Representations: Recently, implicit representations of 3D geometry have gained popularity in learning-based 3D reconstruction [10, 15, 35, 42, 45, 48, 59].”
- 3D Neural Field Generation Using Triplane Diffusion (2022, neutral): “They either solely learn geometry [1,4,5,10,11,15,21,23,45, 47,48,57,65,73] or use posed images to jointly optimize geometry and appearance [6,7,18,25,29,33,36–39, 44, 49, 50, 54, 55 Triplanesareefficient, scalingwiththesurfacearearatherthan volume, and naturally integrate with expressive,…”