Shape As Points: A Differentiable Poisson Solver

By Songyou Peng, Chiyu "Max" Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, Andreas Geiger. Neural Information Processing Systems, 2021

300 citations recorded by the source corpus.

In recent years, neural implicit representations gained popularity in 3D reconstruction due to their expressiveness and flexibility. However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization. In this paper, we revisit the classic yet ubiquitous point cloud representation and introduce a differentiable point-to-mesh layer using a differentiable formulation of Poisson Surface Reconstruction (PSR) that allows for a GPU-accelerated fast solution of the indicator function given an oriented point cloud. The differentiable PSR layer allows us to efficiently and differentiably bridge the explicit 3D point representation with the 3D mesh via the implicit indicator field, enabling end-to-end optimization of surface reconstruction metrics such as Chamfer distance. This duality between points and meshes hence allows us to represent shapes as oriented point clouds, which are explicit, lightweight and expressive. Compared to neural implicit representations, our Shape-As-Points (SAP) model is more interpretable, lightweight, and accelerates inference time by one order of magnitude. Compared to other explicit representations such as points, patches, and meshes, SAP produces topology-agnostic, watertight manifold surfaces. We demonstrate the effectiveness of SAP on the task of surface reconstruction from unoriented point clouds and learning-based reconstruction.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design

View Paper: https://www.semanticscholar.org/paper/1b16ff707bbff4a6ed604db672599645a9060e55

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

Citations in the corpus

MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction (2022, methodology): “More speciﬁcally, we investigate the following architectures: a single, large MLP [49,76,81,82], a dense SDF grid [31], a single feature grid [28,38,53,54] and multi-resolution feature grids [10, 22, 45, 69, 92].”
LION: Latent Point Diffusion Models for 3D Shape Generation (2022, methodology): “Furthermore, training SAP requires signed distance fields (SDFs) for volumetric supervision, which the PointFlow data does not offer.”
Point-E: A System for Generating 3D Point Clouds from Complex Prompts (2022, methodology): “We brieﬂy tried using pre-trained SAP models (Peng et al., 2021) for this purpose, but found that the resulting meshes sometimes lost large portions or important details of the shape that were present in the point clouds.”
Mip-Splatting: Alias-Free 3D Gaussian Splatting (2023, background): “Differentiable point-based rendering methods [20, 36, 39, 43, 49, 53, 57] offer great flexibility in representing intricate structures and are thus well-suited for novel view synthesis.”
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers (2023, background): “However, point clouds, while useful, are not the ideal format for downstream applications requiring 3D content, as converting them to meshes, which apart from being non-trivial [42, 47, 51, 65], can often fail to accurately reflect the characteristics of the underlying mesh datasets.”