Soft 3D reconstruction for view synthesis

By Eric Penner, Li Zhang. ACM Transactions on Graphics, 2017

421 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/45b9f6d454a1b6b47ab5571034463d358c9e4c65

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep blending for free-viewpoint image-based rendering (2018, methodology): “Soft3D [Penner and Zhang 2017] is the most recent IBR algorithm, based on a regular discretization of space using the input images and a sophisticated blending approach using a soft estimation of visibility.”
Plenoxels: Radiance Fields without Neural Networks (2021, methodology): “The most common classical methods for volume rendering are voxel grids [7, 14, 19, 22, 43–45, 53, 54] and multi-plane images (MPIs) [27,34,48,49,58,62].”
Pushing the Boundaries of View Extrapolation With Multiplane Images (2019, background): “Diffuse volumetric scene representations such as the MPI are becoming increasingly popular for view synthesis for a number of reasons: 1) They can represent geometric uncertainty in ambiguous regions as a distribution over depths, thereby trading perceptually-distracting artifacts in those ambiguous”
Neural volumes (2019, methodology): “Penner and Zhang [2017] compute a soft volumetric representation using MVS depth maps to perform novel view synthesis.”
SynSin: End-to-End View Synthesis From a Single Image (2019, methodology): “Traditionally, this was done using depth maps [5, 47] or multi-view geometry [11, 12, 15, 30, 51, 76].”
Baking Neural Radiance Fields for Real-Time View Synthesis (2021, background): “Another popular class of methods uses discretized volumetric representations such as voxel grids [27, 41, 42, 45] or multiplane images [13, 37, 46, 56].”
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes (2020, background): “Numerous other works improve the rendering quality of light fields by exploiting their special structure [16, 57, 63, 75].”
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs (2021, background): “Another line of methods considers voxel grids as scene representation [19, 25, 40, 51, 52, 58] which, however, are typically restricted in terms of their resolution.”
Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans (2020, background): “To extend the range, [62, 6, 45] infer depth maps from input images as proxy geometries.”
DeepVoxels: Learning Persistent 3D Feature Embeddings (2018, background): “In other cases, no 3D reconstruction is necessary [11, 41].”
MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures (2022, background): “Storage can be reduced by interpolating intermediate images with optical flow [5], representing the light field as a neural network [1], or by reconstructing a Multi-Plane Image (MPI) representation of the scene [15,32,37,47,54].”
PlenOctrees for Real-time Rendering of Neural Radiance Fields (2021, background): “Volumebased representations such as voxel grids [40, 17, 23, 13, 53, 42] and multi-plane images (MPIs) [47, 33, 62, 46, 27] are a popular alternative to mesh representations due to their topology-free nature: gradient-based optimization is therefore straightforward, while rendering can still be real”
BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis (2023, methodology): “In this work, we demonstrate how to extract high-quality meshes from a NeRF-like neural volumetric representation.”
NeRF: Representing scenes as neural radiance ﬁelds for view synthesis (neutral): “…sophisticated algorithms over the next two decades; representative papers are [ 8 ] that uses more conventional depth synthesis and warping, and [ 48 ] that acknowledges the limitations of depth and 3D reconstruction, aiming for a notion of a “soft” or “probabilistic” 3D reconstruction that does…”
Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (2023, background): “Penner and Zhang [72] leverage volumetric voxels for continuity in synthesized views and robustness to depth uncertainty.”
IBRNet: Learning Multi-View Image-Based Rendering (2021, methodology): “In more recent work, researchers have proposed improved methods for computing proxy geometry [5, 18], optical flow correction [4, 10, 11], and soft blending [42, 47].”
3D Gaussian Splatting for Real-Time Radiance Field Rendering (2023, methodology): “Volumetric representations for novel-view synthesis were initiated by Soft3D [Penner and Zhang 2017]; deep-learning techniques coupled with volumetric ray-marching were subsequently proposed [Henzler et al. 2019; Sitzmann et al. 2019] building on a continuous differentiable density field to…”
NeRF-: Neural Radiance Fields Without Known Camera Parameters (2021, methodology): “As an alternative, volume-based representations have been proposed to directly model the appearance of the 3D space [Flynn et al. 2016; Mildenhall et al. 2019, 2020; Penner and Zhang 2017; Seitz and Dyer 1999; Sitzmann et al. 2019; Zhou et al. 2018].”