PU-GAN: A Point Cloud Upsampling Adversarial Network

By Ruihui Li, Chi-Wing Fu, Daniel Cohen-Or, P. Heng. IEEE International Conference on Computer Vision, 2019

514 citations recorded by the source corpus.

Point clouds acquired from range scans are often sparse, noisy, and non-uniform. This paper presents a new point cloud upsampling network called PU-GAN, which is formulated based on a generative adversarial network (GAN), to learn a rich variety of point distributions from the latent space and upsample points over patches on object surfaces. To realize a working GAN network, we construct an up-down-up expansion unit in the generator for upsampling point features with error feedback and self-correction, and formulate a self-attention unit to enhance the feature integration. Further, we design a compound loss with adversarial, uniform and reconstruction terms, to encourage the discriminator to learn more latent patterns and enhance the output point distribution uniformity. Qualitative and quantitative evaluations demonstrate the quality of our results over the state-of-the-arts in terms of distribution uniformity, proximity-to-surface, and 3D reconstruction quality.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Geology

View Paper: https://www.semanticscholar.org/paper/119bb19d6a6fc34eaf4e517d0acc102a7cbb4d65

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Philipp Krahenbuhl
2. Xingyi Zhou
3. Tianwei Yin
4. P. Heng
5. Chen Change Loy
6. Bo Dai
7. Jiageng Mao
8. Peng Gao

In Computer Graphics and Computer-Aided Design

1. Philipp Krahenbuhl
2. Tianwei Yin
3. Xingyi Zhou
4. Bo Dai
5. P. Heng
6. Chen Change Loy
7. Hongsheng Li
8. Jun-Yan Zhu

In Geology

1. Tianwei Yin
2. Philipp Krahenbuhl
3. Xingyi Zhou
4. Jiageng Mao
5. Bo Dai
6. Peng Gao
7. Nick Barnes
8. P. Heng

Citations in the corpus

• Unsupervised 3D Shape Completion through GAN Inversion (2021, methodology): “Studies in point cloud upsampling [18, 35] propose some forms of uniformity losses, such as repulsion loss, on point cloud patches.”
• Multimodal Virtual Point 3D Detection (2021, methodology): “Lidar-based methods like PUNet [69], PUGAN [27], and Wang et al. [64] learn high level point-wise features from raw Lidar scans.”
• GRNet: Gridding Residual Network for Dense Point Cloud Completion (2020, methodology): “Following PU-GAN [52], we adopt Uniformity to evaluate the distribution uniformity of the completed point clouds, which can be formulated as”
• Geometric Back-Projection Network for Point Cloud Classification (2019, background): “early work manages to involve the idea of correcting the error with feedback connections in CNNs: e.g., 2D human pose estimation [30], image super-resolution (SR) [31], [32]. In 3D, the recent PU-GAN [33] proposed an error feedback unit achieving feature expansion for point cloud generation (i.e., 3”
• SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer (2021, background): “However, the problem of these folding-based methods [53, 56, 30] is that the 3-dimensional code generated by intermediate layer of network is an implicit representation of target shape, which is hardly interpreted or constrained in order to help refine the shape in local region.”
• Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following (2023, background): “To address this, self-supervised learning has emerged as a promising solution to enhance 3D transfer learning [10, 34, 53, 86].”