Compact 3D Gaussian Representation for Radiance Field

By J. Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, Eunbyung Park. Computer Vision and Pattern Recognition, 2023

472 citations recorded by the source corpus.

Neural Radiance Fields (NeRFs) have demonstrated re-markable potential in capturing complex 3D scenes with high fidelity. However, one persistent challenge that hin-ders the widespread adoption of NeRFs is the computational bottleneck due to the volumetric rendering. On the other hand, 3D Gaussian splatting (3DGS) has recently emerged as an alternative representation that leverages a 3D Gaussisan-based representation and adopts the ras-terization pipeline to render the images rather than volumetric rendering, achieving very fast rendering speed and promising image quality. However, a significant draw-back arises as 3DGS entails a substantial number of 3D Gaussians to maintain the high fidelity of the rendered images, which requires a large amount of memory and stor-age. To address this critical issue, we place a specific emphasis on two key objectives: reducing the number of Gaussian points without sacrificing performance and compressing the Gaussian attributes, such as view-dependent color and covariance. To this end, we propose a learnable mask strategy that significantly reduces the number of Gaussians while preserving high performance. In addition, we propose a compact but effective representation of view-dependent color by employing a grid-based neural field rather than relying on spherical harmonics. Finally, we learn codebooks to compactly represent the geometric attributes of Gaussian by vector quantization. With model compression techniques such as quantization and entropy coding, we consistently show over 25× reduced storage and enhanced rendering speed, while maintaining the quality of the scene representation, compared to 3DGS. Our work provides a comprehensive framework for 3D scene representation, achieving high performance, fast training, compact-ness, and real-time rendering. Our project page is available at https://mainco/d2.github.io/c3dgs/.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9a898d3ebe03a6bd46176b721bc4bb839fe1cdcb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

1. Dejia Xu
2. Zhangyang Wang
3. Zehao Zhu
4. Kairun Wen
5. Kevin Wang
6. Zhiwen Fan

In Computer Vision and Pattern Recognition

1. Dejia Xu
2. Zhangyang Wang
3. Zehao Zhu
4. Kairun Wen
5. Kevin Wang
6. Zhiwen Fan
7. Wenguan Wang
8. Guikun Chen

Citations in the corpus

• Recent advances in 3D Gaussian splatting (2024, unknown): “Vector Quantization, a traditional compression method in signal processing, which involves clustering multi-dimensional data into a finite set of representations, is mainly utilized in Gaussians [40–44].”
• A Survey on 3D Gaussian Splatting (2024, unknown): “FSGS [340] Manuscript submitted to ACM devised a Gaussian Unpooling process for initialization and also introduced depth regularization.”
• LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, background): “Un-structured [37] and structured pruning [1, 27] remove components at weight-level and neuron (mostly channel) levels to provide a smaller network with a smaller or more efficient network architecture.”