Representing Scenes as Neural Radiance Fields for View Synthesis

Subfields: Generative Models, 3D Vision & Pose

Citations in the corpus

• Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, neutral): “This process is analogous to the way that positional information is used to query implicit functions like NeRF [55].”
• Perceiver: General Perception with Iterative Attention (2021, positive): “This is in contrast to the parameterization used in NeRF (Mildenhall et al., 2020), which produces very high frequencies if a moderate number of bands are used (e.g. the 64 th band would have a frequency of 2 64 = 1 .”
• GLIGEN: Open-Set Grounded Text-to-Image Generation (2023, positive): “Similar to encoding boxes, the Fourier embedding [38] can be applied to each keypoint location l = [x, y].”
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020, positive): “A few recent works [27, 44] have experimentally found that a heuristic sinusoidal mapping of input coordinates (called a “positional encoding”) allows MLPs to represent higher frequency content.”
• Learning to Learn with Generative Models of Neural Network Checkpoints (2022, positive): “We project each scalar to a vector representation using a frequency-based encoding scheme (Mildenhall et al., 2020).”
• How to Represent Part-Whole Hierarchies in a Neural Network (2021, positive): “Work on the use of neural fields for generating images has established that there are much better ways to represent the location than using two scalars for its x and y coordinates [Sitzmann et al., 2020, Mildenhall et al., 2020].”
• GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis (2020, unknown): “In this paper, we demonstrate that the dilemma between coarse outputs and entangled latents can be resolved using conditional radiance fields, a conditional variant of a recently proposed continuous representation for novel view synthesis [36].”
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images (2022, unknown): “To synthesize novel views, GRAF utilizes a neural volume rendering approach similar to Nerf [24].”
• Visual Attention Methods in Deep Learning: An In-Depth Survey (2022, unknown): “To compensate for the spatial context, Fourier transform is used to encode the features [140], [141], [142], [143].”
• GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (2023, unknown): “The first column is the rendering result of Shap-E [24] with NeRF [45] as the 3D representation.”
• Fast Dynamic Radiance Fields with Time-Aware Neural Voxels (2022, unknown): “Neural radiance fields are first proposed in [Mildenhall et al. 2020], which models 3D scenes by mapping the coordinate (𝑥,𝑦, 𝑧) and view direction (𝜃, 𝜙) of each point in the space into its color 𝒄 and density 𝜎 .”
• ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation (2023, unknown): “In light of the limited generalizability of direct 3D generative models, a parallel line of research has explored the elevation of 2D diffusion priors into 3D generation, often integrating with 3D representations like NeRF [23].”
• SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis (2023, unknown): “Let a NeRF [27] be a mapping function f that maps a 3D spatial location x and a viewing direction d into a volume density σ and a color value c.”
• Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering (2021, unknown): “We follow the standard metric in [12, 17, 6] by using PSNR, SSIM and LPIPS to measure the rendering quality.”
• Neural RGB-D Surface Reconstruction (2021, unknown): “Instead of computing the weights as probabilities of light reflecting at a given sample point based on the density of the medium [48], we compute weights directly from signed distance values as the product of two sigmoid functions:”
• Dense Depth Priors for Neural Radiance Fields from Sparse Input Views (2021, unknown): “We compare our method to NeRF [21] as well as recent and concurrent work that equally uses sparse depth input in NeRF, namely Depth-supervised NeRF (DS-NeRF) [10] and NerfingMVS [26].”
• Volume Rendering of Neural Implicit Surfaces (2021, unknown): “In previous neural volumetric rendering approaches [21, 15, 39], the density function, σ, was modeled with a general-purpose Multi-Layer Perceptron (MLP).”
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes (2021, unknown): “Although direct comparisons with neural volumetric rendering methods are not possible, we report frametimes over 500× faster than NeRF [32] and 50× faster than NSVF [24] in similar experimental settings.”