Urban Radiance Fields

By Konstantinos Rematas, An Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasacchi, Thomas A. Funkhouser, Vittorio Ferrari. Computer Vision and Pattern Recognition, 2021

344 citations recorded by the source corpus.

The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e.g., Street View). Given a sequence of posed RGB images and lidar sweeps acquired by cameras and scanners moving through an outdoor scene, we produce a model from which 3D surfaces can be extracted and novel RGB images can be synthesized. Our approach extends Neural Radiance Fields, which has been demonstrated to synthesize realistic novel images for small scenes in controlled settings, with new methods for leveraging asynchronously captured lidar data, for addressing exposure variation between captured images, and for leveraging predicted image segmentations to supervise densities on rays pointing at the sky. Each of these three extensions provides significant performance improvements in experiments on Street View data. Our system produces state-of-the-art 3D surface reconstructions and synthesizes higher quality novel views in comparison to both traditional methods (e.g. COLMAP) and recent neural representations (e.g. Mip-NeRF).

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Environmental Engineering

View Paper: https://www.semanticscholar.org/paper/102d29870ba101004afce311823df85a9f304be7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering (2021, background): “NeRF has inspired many subsequent works that extend its continuous neural volumetric representation for more practical scenarios beyond simple static scenes, including unbounded scenes [42,3], dynamic scenes [12,41], nonrigidly deforming objects [27,25,26,38], phototourism settings with changing ill”
DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (2023, unknown): “Our method achieves the optimal quality with less rendering time compared to those of NeRF-based methods designed for unbounded scenes(e.g., Mip-NeRF [1], Mip-NeRF360 [2], Urban-NeRF [12]).”
Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation (2022, methodology): “Unlike methods such as [23] that force the network to learn a negative truncation value−tr for points behind the truncation region, we simply mask out these points during rendering to avoid solving surface intersection ambiguities [36] as proposed in [1].”
NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “Neural Radiance Fields were ﬁrst proposed by Mildenhall et al. [1] in 2020 for novel view synthesis.”
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly- Throughs (2021, background): “Concurrent to us, Urban Radiance Fields [30] (URF), CityNeRF [43], and BlockNeRF [34] target urban-scale environments.”
A Survey on 3D Gaussian Splatting (2024, unknown): “The performance scores are primarily sourced from the original papers, except where indicated otherwise.”
UniSim: A Neural Closed-Loop Sensor Simulator (2023, unknown): “ing highly photorealistic sensor observations [1, 35, 43, 50, 55, 56, 63, 64].”