K-Planes: Explicit Radiance Fields in Space, Time, and Appearance

By Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, B. Recht, Angjoo Kanazawa. Computer Vision and Pattern Recognition, 2023

967 citations recorded by the source corpus.

We introduce k-planes, a white-box model for radiance fields in arbitrary dimensions. Our model uses planes to represent a d-dimensional scene, providing a seamless way to go from static (d = 3) to dynamic (d= 4) scenes. This planar factorization makes adding dimension-specific priors easy, e.g. temporal smoothness and multi-resolution spatial structure, and induces a natural decomposition of static and dynamic components of a scene. We use a linear feature decoder with a learned color basis that yields similar performance as a nonlinear black-box MLP decoder. Across a range of synthetic and real, static and dynamic, fixed and varying appearance scenes, k-planes yields competitive and often state-of-the-art recon- struction fidelity with low memory usage, achieving 1000x compression over a full 4D grid, and fast optimization with a pure PyTorch implementation. For video results and code, please see sarafridov.github.io/K-Planes.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2b4de48703d5278afbce69844d5ed92b5a699ee1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (2023, methodology): “[3, 8, 31] represent further advancements in faster dynamic scene learning.”
Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, background): “Various spatial structures have been explored in previous NeRF-based representations, including dense voxel grids [20], [22], sparse voxel grids [17], [21], point clouds [40], multiple compact low-rank tensor components [23], [41], [42], and multi-resolution hash tables [24].”
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, background): “(b) Methods that represent the scene using an Eulerian representation on a 4D space-time grid, often with various grid decompositions for efficiency such as planar decomposition or hash functions [5, 10, 36].”
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields (2023, methodology): “Triplane [5], TensoRF [7], K-Plane [10], TILED [47] adopt tensor factorization to obtain efficient explicit representation.”
Compact 3D Gaussian Representation for Radiance Field (2023, background): “Subsequent research efforts have been directed toward the reduction of the memory footprint while maintaining or even enhancing the performance quality by grid factorization [6, 7, 11, 13, 15, 31], hash grids [30], grid quantization [38, 40] or pruning [10, 35].”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, neutral): “A KL divergence term is added to minimize the divergence of the ray’s distribution and this noisy depth distribution (See [22] for details).”
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (2023, methodology): “Drawing inspiration from seminal NeRF acceleration research, numerous studies on dynamic scene modeling have integrated discrete structures, such as voxel-grids [11, 38], or planes [6, 36].”
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering (2023, methodology): “K-planes [14] used neural planes to parameterize a 3D scene, optionally with an additional temporal plane to accommodate dynamics.”
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (2024, methodology): “While some methods [36, 38, 49] achieve faster inference speed with grid-based backbones [13, 26, 37], they often suffer from trade-offs, leading to either high training costs or compromised rendering quality.”
GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (2023, methodology): “A common paradigm to model a dynamic scene is directly storing it in an MLP with 4D coordinates as the input [9, 44] or a 4D tensor with the time dimension or spatial dimensions compressed [1, 4, 7, 38].”
Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (2023, methodology): “, 2023), including their low-rank decompositions (Fridovich-Keil et al., 2023; Cao & Johnson, 2023; Attal et al., 2023), to learn a 6D plenoptic function without explicitly modeling scene motion.”
HexPlane: A Fast Representation for Dynamic Scenes (2023, methodology): “Our paper is most closely related to K-Planes [14], which also employs six feature planes for representation.”
VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (2024, neutral): “…becomes a popular 3D representation for photo-realistic novel-view synthesis in recent years [35], many variants are proposed to improve quality [2–4, 24, 45, 47–49, 57], increase speed [8, 9, 11, 14, 20, 32, 36, 37, 40, 43, 46, 58, 60], extend to dynamic scenes [7, 15, 18, 25, 27, 50], and so on.”
Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, methodology): “State-of-the-art dynamic view synthesis meth-ods [4, 12, 28, 48, 80, 87] adopt a holistic approach where multiple frames are represented in a single model.”
Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering (2022, methodology): “In addition, there are also several concurrent works [4, 12, 20] adopt 6-plane decomposition for dynamic scenes.”
A Survey on 3D Gaussian Splatting (2024, neutral): “An explicit radiance field directly represents the distribution of light in a discrete spatial structure, such as a voxel grid or a set of points [57, 236].”
Text-to-3D using Gaussian Splatting (2023, background): “After works have emerged to improve NeRF in reconstruction quality [7, 8, 76], handling large-scale [13, 40, 68, 81] and dynamic scenes [3, 47, 51, 60, 73], improving training [10, 44, 67, 79], rendering [25, 56, 80] speed and facilitating down-stream tasks [17, 46, 78, 83, 84].”