Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering

By Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, Bo Dai. Computer Vision and Pattern Recognition, 2023

863 citations recorded by the source corpus.

Neural rendering methods have significantly advanced photo-realistic 3D scene rendering in various academic and industrial applications. The recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed combining the benefits of both primitive-based representations and volumetric representations. However, it often leads to heavily redundant Gaussians that try to fit every training view, neglecting the underlying scene ge-ometry. Consequently, the resulting model becomes less robust to significant view changes, texture-less area and lighting effects. We introduce Scaffold-GS, which uses an-chor points to distribute local 3D Gaussians, and predicts their attributes on-the-fly based on viewing direction and distance within the view frustum. Anchor growing and pruning strategies are developed based on the importance of neural Gaussians to reliably improve the scene cover-age. We show that our method effectively reduces redun-dant Gaussians while delivering high-quality rendering. We also demonstrates an enhanced capability to accommodate scenes with varying levels-of-detail and view-dependent ob-servations, without sacrificing the rendering speed. Project page: https://city-super.github.iolscaffold-gsl.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a294b8632fed59e7079ef6187b0afa532c97ed7f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Bo Dai
2. Linning Xu
3. Dejia Xu
4. Zhangyang Wang
5. Zehao Zhu
6. Tao Lu
7. Mulin Yu
8. Zhiwen Fan

In Computer Graphics and Computer-Aided Design

1. Bo Dai
2. Linning Xu
3. Dejia Xu
4. Zhangyang Wang
5. Lihan Jiang
6. Zehao Zhu
7. Kairun Wen
8. Kevin Wang

In Computer Vision and Pattern Recognition

1. Bo Dai
2. Linning Xu
3. Dejia Xu
4. Zhangyang Wang
5. Lihan Jiang
6. Zehao Zhu
7. Kairun Wen
8. Kevin Wang

Citations in the corpus

• LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, methodology): “Datasets and Metrics Comparisons are performed on the widely adopted scene-scale view synthesis data created by Mip-NeRF360 [6], which contains nine real-world large-scale scenes (5 unbounded outdoors, 4 indoors with detailed backgrounds).”
• Structured 3D Latents for Scalable and Versatile 3D Generation (2024, neutral): “More Implementation Details”
• Recent advances in 3D Gaussian splatting (2024, neutral): “In terms of disk data storage, SASCGS [42] utilizes the entropy encoding method DEFLATE, which utilizes a combination of the LZ77 algorithm and Huffman coding, to compress the data.”
• A Survey on 3D Gaussian Splatting (2024, neutral): “Due to the discrete sampling paradigm (viewing each pixel as a single point instead of an area), 3D GS is susceptible to aliasing when dealing with varying resolutions, which leads to blurring or jagged edges.”
• PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, neutral): “Restrictions apply.”
• Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, methodology): “With such strengths, it has significantly accelerated the process of replicating the real world into a digital counterpart [6]– [9], igniting the community’s imagination for scaling real-to-simulation environments [3], [10], [11].”