GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces

By Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, Yuexin Ma. arXiv.org, 2023

307 citations recorded by the source corpus.

The advent of neural 3D Gaussians has recently brought about a revolution in the field of neural rendering, facilitating the generation of high-quality renderings at real-time speeds. However, the explicit and discrete representation encounters challenges when applied to scenes featuring reflective surfaces. In this paper, we present GaussianShader, a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces while preserving the training and rendering efficiency. The main challenge in applying the shading function lies in the accurate normal estimation on discrete 3D Gaussians. Specifically, we proposed a novel normal estimation framework based on the shortest axis directions of 3D Gaussians with a delicately designed loss to make the consistency between the normals and the geometries of Gaussian spheres. Experiments show that GaussianShader strikes a commendable balance between efficiency and visual quality. Our method surpasses Gaussian Splatting in PSNR on specular object datasets, exhibiting an improvement of 1.57dB. When compared to prior works handling reflective surfaces, such as Ref-NeRF, our optimization time is significantly accelerated (23h vs. 0.58h). Please click on our project website to see more results.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9420a5e441125271d837983e0532ad7c4f76efa6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Xiaoxiao Long
2. Yuanchen Guo
3. Yuan Liu
4. Lingjie Liu

In Computer Graphics and Computer-Aided Design

1. Yuanchen Guo
2. Yuan Liu
3. Lingjie Liu
4. Jonathan T. Barron

In Computer Vision and Pattern Recognition

1. Yuan Liu
2. Marc Habermann
3. Xiaoxiao Long
4. Yuanchen Guo
5. Song-Hai Zhang
6. Yuexin Ma
7. Lingjie Liu
8. Jonathan T. Barron

Citations in the corpus

• A Survey on 3D Gaussian Splatting (2024, unknown): “In SLAM , [140, 203], where environmental dynamics demand real-time map updates, requires explicit modeling of dynamic adjustments ( e.g ., object motions and interactions), material compliance, etc .”
• Recent advances in 3D Gaussian splatting (2024, unknown): “The latter directly converts 3D Gaussian into 4D Gaussian and enhances the temporal continuity of SyncDreamer [192] by fusing spatial volumes at adjacent timestamps, achieving time synchronization to generate better cross-time multi-view images for supervision.”
• PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, unknown): “As discussed in previous methods [21], [22], Gaussians often do not conform well to actual surfaces, resulting in poor geometric accuracy.”
• Wonder3D: Single Image to 3D Using Cross-Domain Diffusion (2023, background): “…3D geometry from a single image [13, 26, 33, 38, 40, 42, 45] stands as a fundamental task in computer graphics and 3D computer vision, benefiting a wide range of versatile applications such as novel view synthesis [7, 25, 35], 3D content creation [36, 48], and robotics grasping [29, 79].”