A Papier-Mache Approach to Learning 3D Surface Generation

By Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry. 2018

1,334 citations recorded by the source corpus.

We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues. We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) autoencoding shapes, and (ii) single-view reconstruction from a still image. We also provide results showing its potential for other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In 3D Vision & Pose

Citations in the corpus

Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, methodology): “Moreover, they are only able to generate meshes with simple topology [70], require a reference template from the same object class [33, 41, 57] or cannot guarantee closed surfaces [26].”
Learning to Reconstruct Shapes from Unseen Classes (2018, methodology): “We further compare how our model, AtlasNet, and Shin et al. [2018] perform on real images from Pix3D.”
Learning Implicit Fields for Generative Shape Modeling (2018, methodology): “We compare our approach with two state-of-the-art SVR methods, HSP [18], an octree-based method that uses 3D CNN decoders to generate 256(3) voxels, and AtlasNet [16], which warps surface patches onto target shapes.”
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling (2018, neutral): “144 AtlasNet [18] N/A 0.”
Diffusion Probabilistic Models for 3D Point Cloud Generation (2021, methodology): “We evaluate the reconstruction quality of the proposed auto-encoder, with comparisons against state-of-the-art point cloud auto-encoders: AtlasNet [10], PointFlow [25] and ShapeGF [2].”
Learning Gradient Fields for Shape Generation (2020, positive): “AtlasNet [6] The code we used for the AtlasNet decoder comes from the official code repository2.”
Self-supervised Learning of 3D Objects from Natural Images (2019, background): “One research direction is how neural networks handle irregular 3D representations, such as high-resolution voxels [11, 37], point clouds [7], meshes [9, 43] and implicit functions [3, 24, 26].”
LION: Latent Point Diffusion Models for 3D Shape Generation (2022, neutral): “[99] Thibault Groueix, Matthew Fisher, Vladimir G.”
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction (2019, methodology): “[11] present AtlasNet to generate surfaces of 3D shapes using a set of parametric surface elements.”
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis (2020, background): “Key advantages over voxel [6,9,50,52, 65,66] or mesh-based [15,26,43,62] methods are that they do not discretize space and are not restriced in topology.”
What Do Single-View 3D Reconstruction Networks Learn? (2019, methodology): “10 (right), where AtlasNet trails all other approaches.”
Shape As Points: A Differentiable Poisson Solver (2021, methodology): “Shape-As-Points produces higher quality geometry compared to other explicit representations [11,17,20,64] and requires signiﬁcantly less inference time for extracting geometry compared to neural implicit representations [42]. functions, and meshes allows us to represent shapes as oriented point…”
Im2Vec: Synthesizing Vector Graphics without Vector Supervision (2021, methodology): “Deep learning techniques for parametric vector shapes have recently garnered signiﬁcant interest from the machine learning community [19, 11, 13, 40, 27].”
Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve (2020, background): “Atthe same time, we are now seeing a variety of advances in understanding the shape of a single object from image view(s), driven by exploration of various geometric representations: voxels [6,40,43], points [10,44], meshes [8,14,41], and implicit surfaces [30,32].”
3D Shape Generation and Completion through Point-Voxel Diffusion (2021, background): “Many prior works have explored point cloud generation in terms of autoencoding [1, 11, 47], single-view reconstruction [12, 10, 18, 17], and adversarial generation [34, 48, 40].”
Texture Fields: Learning Texture Representations in Function Space (2019, background): “In recent years, 3D reconstruction of shapes from single images has made great progress [6, 7, 11, 21, 38].”
Learning Shape Templates With Structured Implicit Functions (2019, methodology): “We argue that our structured implicit representation is superior for template learning compared to decoding voxels [6, 54, 55], sparsevoxel octrees [49], points [11], meshes [15, 21, 52], box primitives [50], signed-distance function estimators [38], or indicator function estimators [32].”