PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

By Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge J. Belongie, Bharath Hariharan. 2019

831 citations recorded by the source corpus.

As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show that our model can faithfully reconstruct point clouds and learn useful representations in an unsupervised manner. The code is available at https://github.com/stevenygd/PointFlow.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Geology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Geology

In Generative Models

Citations in the corpus

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “, 2020) to 3D point cloud generation (Yang et al., 2019), light-transport simulation (Müller et al.”
Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “So far, flows have been applied to graphs (Deng et al., 2019), molecules (Madhawa et al., 2019; Honda et al., 2019), point clouds (Yang et al., 2019), and part models for motion synthesis (Henter et al., 2019).”
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (2020, methodology): “We use a series of the gate-bias modulation (Grathwohl et al. 2018; Yang et al. 2019) (called ”ConcatSquash”) networks to model the function ϕ of the conditional continuous normalizing ows.”
Diffusion Probabilistic Models for 3D Point Cloud Generation (2021, methodology): “We evaluate the reconstruction quality of the proposed auto-encoder, with comparisons against state-of-the-art point cloud auto-encoders: AtlasNet [10], PointFlow [25] and ShapeGF [2].”
Learning Gradient Fields for Shape Generation (2020, positive): “[64] and Section 1, both of these approaches lead to several drawbacks.”
VFlow: More Expressive Generative Flows with Variational Data Augmentation (2020, background): “, 2019), 3D point cloud generation (Yang et al., 2019), semi-supervised learning (Nalisnick et al.”
3D Shape Generation and Completion through Point-Voxel Diffusion (2021, methodology): “GAN-based models and flow-based models [21, 1, 46, 16, 17] also adopt a probabilistic view but separate shape-level distribution from point-level distribution.”
WaveNODE: A Continuous Normalizing Flow for Speech Synthesis (2020, methodology): “Popular CNF-based models often employ an MBN layer for stable training while constructing a deep architecture (Grathwohl et al., 2018; Yang et al., 2019).”
Shape As Points: A Differentiable Poisson Solver (2021, background): “Explicit shape representations and learning algorithms depending on such representations directly parameterize the surface of the geometry, either as a point cloud [17, 38, 53, 54, 64, 67], parameterized mesh [22, 24, 27, 61] or surface patches [2, 20, 37, 43, 65, 68, 69].”
Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve (2020, methodology): “At the same time, we are now seeing a variety of advances in understanding the shape of a single object from image view(s), driven by exploration of various geometric representations: voxels [6,40,43], points [10,44], meshes [8,14,41], and implicit surfaces [30,32].”
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation (2023, background): “, point cloud [2, 60, 100, 108], octree [38], and implicit representations [14, 56].”
Low-Light Image Enhancement with Normalizing Flow (2021, background): “2019) generates faces with specific attributes, (Pumarola et al. 2020; Yang et al. 2019) use conditional flow to generate point clouds.”
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs (2021, background): “1 INTRODUCTION 1 In the past decade, a plethora of deep generative models has been developed for various domains such as images (Karras et al., 2019; Razavi et al., 2019), audio (Oord et al., 2016a; Kong et al., 2021), point clouds (Yang et al., 2019) and graphs (De Cao & Kipf, 2018).”
LION: Latent Point Diffusion Models for 3D Shape Generation (2022, methodology): “Table 1: Generation metrics (1-NNA↓) on airplane, chair, car categories from ShapeNet dataset from PointFlow [31].”
SRFlow: Learning the Super-Resolution Space with Normalizing Flow (2020, background): “Recent works have investigated conditional flow models for point cloud generation [37,51] as well as class [25] and image [3,49] conditional generation of images.”
Shap-E: Generating Conditional 3D Implicit Functions (2023, background): “[69] likewise train a point cloud auto-encoder, but their decoder is itself a conditional generative model (i.”
Accelerating 3D deep learning with PyTorch3D (2019, background): “Generative models [61, 62, 39] might one day aid artists in authoring 3D content.”
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World (2020, methodology): “[45] transforms Gaussian-3D samples into a point cloud shape conditioned on class via normalizing flows, and [4] uses VAEs and GANs to reconstruct LiDAR from noisy samples.”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, methodology): “Recent architectures apply kernel point convolutions [67], tree-based graph convolutions [60], and normalizing flows [77].”