Representation Learning and Adversarial Generation of 3D Point Clouds

By Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, Leonidas J. Guibas. 2017

92 citations recorded by the source corpus.

Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling. In this paper, we look at geometric data represented as point clouds. We introduce a deep autoencoder network for point clouds, which outperforms the state of the art in 3D recognition tasks. We also design GAN architectures to generate novel point-clouds. Importantly, we show that by training the GAN in the latent space learned by the autoencoder, we greatly boost the GAN's data-generating capacity, creating significantly more diverse and realistic geometries, with far simpler architectures. The expressive power of our learned embedding, obtained without human supervision, enables basic shape editing applications via simple algebraic manipulations, such as semantic part editing and shape interpolation.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Daniel Cohen-Or
2. Chi-Wing Fu
3. P. Heng
4. Michael M. Bronstein
5. Or Litany
6. A. Bronstein
7. Chen Feng
8. Yaoqing Yang

In Computer Graphics and Computer-Aided Design

1. Daniel Cohen-Or
2. P. Heng
3. Chi-Wing Fu
4. Or Litany
5. Jonathan Sauder
6. Bjarne Sievers
7. Andreas Geiger
8. Michael M. Bronstein

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Andreas Geiger
3. Dacheng Tao
4. Jitendra Malik
5. Ming-Hsuan Yang
6. Xiaogang Wang
7. Trevor Darrell
8. Andrew Zisserman

Citations in the corpus

• RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials (2018, background): “Learning-based 3D Reconstruction: The development of large 3D shape databases [4, 38] has fostered the development of learning based solutions [3, 5, 8, 37] to 3D reconstruction.”
• Deformable Shape Completion with Graph Convolutional Autoencoders (2017, background): “The application addressed in this paper is a very active research area in computer vision and graphics, ranging from completion of small holes [48] and larger missing regions in individual objects [1, 45, 55, 61, 49], to entire scenes [51].”
• Self-Supervised Deep Learning on Point Clouds by Reconstructing Space (2019, methodology): “In the first experiment, we follow the same procedure as in [1, 34, 37, 12].”
• AmbientGAN: Generative models from lossy measurements (2018, background): “(2016)], and 3D models [Achlioptas et al. (2017); Wu et al. (2016)]. A learned generative model can be useful for many applications. A string of papers [Bora et al. (2017); Zhu et al. (2016); Yeh et al.”
• PU-Net: Point Cloud Upsampling Network (2018, background): “Achlioptas et al. [2] introduce a deep auto-encoder network for processing 3D point clouds.”
• FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation (2017, methodology): “The only work that we are aware of on end-to-end deep auto-encoder that directly handles point clouds is [1].”