Differentiable surface splatting for point-based geometry processing

By Yifan Wang, Felice Serena, Shihao Wu, Cengiz Öztireli, Olga Sorkine-Hornung. ACM Transactions on Graphics, 2019

392 citations recorded by the source corpus.

We propose Differentiable Surface Splatting (DSS), a high-fidelity differentiable renderer for point clouds. Gradients for point locations and normals are carefully designed to handle discontinuities of the rendering function. Regularization terms are introduced to ensure uniform distribution of the points on the underlying surface. We demonstrate applications of DSS to inverse rendering for geometry synthesis and denoising, where large scale topological changes, as well as small scale detail modifications, are accurately and robustly handled without requiring explicit connectivity, outperforming state-of-the-art techniques. The data and code are at https://github.com/yifita/DSS.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/15a8a5acbe771477f83366bcd60897894c3cdb9b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

3D Gaussian Splatting for Real-Time Radiance Field Rendering (2023, methodology): “…computes the color𝐶 of a pixel by blendingN ordered points overlapping the pixel: 𝐶 = ∑︁ 𝑖∈N 𝑐𝑖𝛼𝑖 𝑖−1∏ 𝑗=1 (1 − 𝛼 𝑗 ), (3) where c𝑖 is the color of each point and 𝛼𝑖 is given by evaluating a 2D Gaussian with covariance Σ [Yifan et al. 2019] multiplied with a learned per-point opacity.”
Compact 3D Gaussian Representation for Radiance Field (2023, methodology): “To mitigate this, point-based neural rendering methods have been proposed, processing the points through rasterization-based point splatting and differentiable raster-ization [22, 43, 46].”
A Survey on 3D Gaussian Splatting (2024, neutral): “The innovation of 3D GS lies in its unique blend of the benefits of differentiable pipelines and point-based rendering techniques [197, 210, 297, 349].”
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting (2023, positive): “For rendering new viewpoints, the method of splatting, as described in [54], is utilized for positioning the Gaussians on the camera planes.”
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (2023, methodology): “Unlike implicit representation methods such as NeRF [45], which renders images based on volume rendering, 3D-GS renders images through splatting [84], achieving real-time speed.”
SynSin: End-to-End View Synthesis From a Single Image (2019, methodology): “This method is related to the point cloud rasterisers of [1, 24, 70].”
LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, methodology): “When rendering 2D images from the 3D Gaussians, the technique of splatting [36, 65] is employed for the Gaus-sians within the camera planes.”
Mip-Splatting: Alias-Free 3D Gaussian Splatting (2023, background): “Differentiable point-based rendering methods [20, 36, 39, 43, 49, 53, 57] offer great flexibility in representing intricate structures and are thus well-suited for novel view synthesis.”
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting (2024, background): “Early efforts adopt CNNs to estimate neural points [2,66,71] or multi-plane images (MPIs) [33,39,76], achieving rendering via point splatting or alpha compositing.”
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting (2023, background): “It represents the scene as a combination of 3D Gaussians with attributes for modeling intricate shape and appearances, and 2D images are rendered via splatting-based rasterization[57].”
Accelerating 3D deep learning with PyTorch3D (2019, background): “DSS [63] renders each point as a disk.”
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (2023, background): “However, since 2D surfels are discontinuous, they need careful regularization to prevent holes [33, 50].”
Gaussian Grouping: Segment and Edit Anything in 3D Scenes (2023, methodology): “Refer to [55], we compute α ′ i by measuring a 2D Gaussian with covariance Σ 2D multiplied with a learned per-point opacity α i , and where Σ 3D is the 3D covariance matrix, Σ 2D is the splatted 2D version [61].”
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering (2023, background): “Traditional primitive-based representations like meshes and points [6, 26, 32, 55] are faster due to the use of rasterization techniques tailored for modern GPUs.”
2D Gaussian Splatting for Geometrically Accurate Radiance Fields (2024, methodology): “Differentiable point-based rendering [Aliev et al. 2020; Insafutdinov and Dosovitskiy 2018; Rückert et al. 2022; Wiles et al. 2020; Yifan et al. 2019]has been explored extensively due to its efficiency and flexibility in representing intricate structures.”
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance (2020, background): “…casting/tracing [41, 33, 24, 26, 38, 27], or rasterization [28, 20, 10, 25, 4], while popular models to represent 3D geometry include point clouds [49], triangle meshes [4], implicit representations deﬁned over volumetric grids [17], and recently also neural implicit representations, namely, zero…”