3D Gaussian Splatting for Real-Time Radiance Field Rendering
By Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, George Drettakis. ACM Transactions on Graphics, 2023
9,441 citations recorded by the source corpus.
Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.
Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/2cc1d857e86d5152ba7fe6a8355c2a0150cc280a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
- Andreas Geiger
- Jonathan T. Barron
- Ben Mildenhall
- Pratul P. Srinivasan
- Matthew Tancik
- Vincent Sitzmann
- Matthias Nießner
- Zexiang Xu
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Andrea Vedaldi
- Trevor Darrell
- Jitendra Malik
- Chen Change Loy
Citations in the corpus
- gsplat: An Open-Source Library for Gaussian Splatting (2024, positive): “Specifically, we compute the affine transform J ∈ R 2 × 3 as: Note, unlike the original implementation by Kerbl et al. (2023), we do not use the OpenGL NDC coordinate system as an intermediate representation.”
- Recent advances in 3D Gaussian splatting (2024, neutral): “…up both training and rendering, instead of predicting density values and colors for all sample points with neural networks, 3D Gaussian Splatting [5] abandons the neural network and directly optimizes Gaussian ellipsoids to which attributes like position P , rotation R , scale S , opacity α…”
- PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (2023, methodology): “2), a unified simulation-rendering framework for generative dynamics based on continuum mechanics and 3D GS. Adopted from Kerbl et al. [16], we first reconstruct a GS representation of a static scene, with an optional anisotropic loss term to regularize over-skinny kernels.”
- A Survey on 3D Gaussian Splatting (2024, neutral): “5), shares the same image formation model as NeRF-style volumetric rendering (Eq.”
- Structured 3D Latents for Scalable and Versatile 3D Generation (2024, positive): “For the versatile decoding of SL AT , we implement decoders for various 3D representations, namely D GS for 3D Gaussians [13], D RF for Radi-ance Fields [20], and D M for meshes.”
- SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes (2023, methodology): “It enables fast model training and real-time inference, achieving state-of-the-art (SOTA) visual quality.”
- NeRF: Representing scenes as neural radiance fields for view synthesis (positive): “…], RefNeRF at CVPR 22 [ 73 ], our own work on Neural Implicit Evolution at ECCV 22 [ 39 ], InstantNGP from SIGGRAPH 22 [ 43 ], Dynamic image-based rendering from CVPR 23 [ 33 ], 3D Gaussian Splatting for Real-Time Radiance Field Rendering from SIG-GRAPH 23 [ 24 ], and DreamFusion at ICLR 23 [ 50 ].”
- NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, positive): “The model achieved good results on the CelebA dataset [115], the CARLA dataset [116], and the ShapeNet subclasses from SRN [119], but is outperformed by its backbone π -GAN for conditional image synthesis.”
- GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting (2023, positive): “GS (Gaussian Splatting) [15] represents an explicit 3D scene using point clouds, where Gaussians are employed to depict the scene’s structure.”
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (2023, methodology): “To compute p i ( u, v ) in the image space, we linearize the perspective transformation as in Zwicker et al. (2002); Kerbl et al. (2023).”
- Gaussian Splatting SLAM (2023, positive): “In this paper, we present the first online visual SLAM system based solely on the 3D Gaussian Splatting (3DGS) representation [10] recently making a big impact in offline scene reconstruction.”
- VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (2024, positive): “Recently, 3D Gaussian Splatting (3DGS) [21] emerges as a promising approach with impressive performance in visual quality and rendering speed, enabling photo-realistic and real-time rendering at 1080p resolution.”
- Depth Anything V2 (2024, neutral): “Precise depth information is not only favorable in classical applications, such as 3D reconstruction [47, 32, 93], navigation [82], and autonomous driving [80], but is also preferable in modern scenarios, e . g ., AI-generated content, including images [101], videos [39], and 3D scenes [87, 64, 68].”
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, methodology): “Following [41], we optimize these parameters through differentiable splatting and gradient-based backpropagation, and interleave with density control of Gaussians.”
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, methodology): “We can render out depth-maps for any view by using the Max [24] rendering equation for Gaussians [17] but replacing the colour component of each Gaussian with the depth of that Gaussian’s center, as seen in Figure 1.”
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, methodology): “Recently, 3D Gaussian Splatting (3D-GS) [32], combining the idea of point-based rendering and splatting techniques [72] for rendering, achieves real-time speed with comparable rendering quality to the best MLP-based renderer, Mip-NeRF 360 [5].”
- LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation (2024, methodology): “Another promising 3D representation is Gaussian splatting [17] for its expressiveness and efficient rendering capabilities.”
- Depth Anything 3: Recovering the Visual Space from Any Views (2025, neutral): “Feed-Forward Novel View Synthesis Novel view synthesis (NVS) has long been a core problem in computer vision and graphics [8, 34, 49], and interest has increased with the rise of neural rendering [28, 44, 57, 77, 78].”