FreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization

By Jiawei Yang, Marco Pavone. Computer Vision and Pattern Recognition, 2023

444 citations recorded by the source corpus.

Novel view synthesis with sparse inputs is a challenging problem for neural radiance fields (NeRF). Recent efforts alleviate this challenge by introducing external supervision, such as pre-trained models and extra depth signals, or by using non-trivial patch-based rendering. In this paper, we present Frequency regularized NeRF (FreeNeRF), a surprisingly simple baseline that outperforms previous methods with minimal modifications to plain NeRF. We analyze the key challenges in few-shot neural rendering and find that frequency plays an important role in NeRF's training. Based on this analysis, we propose two regularization terms: one to regularize the frequency range of NeRF's inputs, and the other to penalize the near-camera density fields. Both techniques are “free lunches” that come at no additional computational cost. We demonstrate that even with just one line of code change, the original NeRF can achieve similar performance to other complicated methods in the few-shot setting. FreeNeRF achieves state-of-the-art performance across diverse datasets, including Blender, DTU, and LLFF. We hope that this simple baseline will motivate a rethinking of the fundamental role of frequency in NeRF's training, under both the low-data regime and beyond. This project is released at FreeNeRF.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/59015e5b1f0fbe6a41feb87b1fbf76f3c7339840

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis (2023, unknown): “FreeNeRF [61] potentially complements SparseNeRF.”
CAT3D: Create Anything in 3D with Multi-View Diffusion Models (2024, background): “…developed specialized solutions for different input settings, such as geometry regularization techniques targeted for sparse-view reconstruction [4, 5], feed-forward models trained to create 3D objects from single images [6], or the use of image-conditioned [7] or text-conditioned [8]…”
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (2024, methodology): “After applying depth regularization, DNGaussian successfully recovers accurate geometry and synthesizes high-quality novel views. tic appearance and accurate geometry from just a handful of input views [5, 16, 27, 35, 36, 42, 49, 52, 54].”
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting (2023, methodology): “We compare our method with 3DGS [26], FreeNeRF [55] and SparseN-eRF [50].”
Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, unknown): “Several works proposed to address these issues through regularization using geometric priors [10, 38, 43, 48, 54, 63, 70, 75], which, however, are sensitive to noise in the estimated depth or normals.”
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (2023, background): “…through the use of MLPs. Subsequent research has expanded the utility of NeRF to various applications, encompassing tasks such as mesh reconstruction from a collection of images [20, 45], inverse rendering [5, 25, 54], optimization of camera parameters [21, 47, 48], and few-shot learning [10, 51].”