NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction

By Yiming Wang, Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, Lingjie Liu. IEEE International Conference on Computer Vision, 2022

422 citations recorded by the source corpus.

Recent methods for neural surface representation and rendering, for example NeuS [59], have demonstrated the remarkably high-quality reconstruction of static scenes. However, the training of NeuS takes an extremely long time (8 hours), which makes it almost impossible to apply them to dynamic scenes with thousands of frames. We propose a fast neural surface reconstruction approach, called NeuS2, which achieves two orders of magnitude improvement in terms of acceleration without compromising reconstruction quality. To accelerate the training process, we parameterize a neural surface representation by multi-resolution hash encodings and present a novel lightweight calculation of second-order derivatives tailored to our networks to leverage CUDA parallelism, achieving a factor two speed up. To further stabilize and expedite training, a progressive learning strategy is proposed to optimize multi-resolution hash encodings from coarse to fine. We extend our method for fast training of dynamic scenes, with a proposed incremental training strategy and a novel global transformation prediction component, which allow our method to handle challenging long sequences with large movements and deformations. Our experiments on various datasets demonstrate that NeuS2 significantly outperforms the state-of-the-arts in both surface reconstruction accuracy and training speed for both static and dynamic scenes. The code is available at our website: https://vcai.mpi-inf.mpg.de/projects/NeuS2/.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/781120c76e42cd13586b01311647815f5c0b0f52

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

2D Gaussian Splatting for Geometrically Accurate Radiance Fields (2024, positive): “Moreover, the training and representational power of NeRF have been enhanced using feature-grid based scene representations [Chen et al. 2022, 2023c; Fridovich-Keil et al. 2022; Liu et al. 2020; Müller et al. 2022; Sun et al. 2022a].”
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (2023, neutral): “3.2.”
Neuralangelo: High-Fidelity Neural Surface Reconstruction (2023, background): “Follow-up works extend the above approaches to realtime at the cost of surface fidelity [18, 37], while others [3, 5, 44] use auxiliary information to enhance the reconstruction results.”
VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (2024, neutral): “…becomes a popular 3D representation for photo-realistic novel-view synthesis in recent years [35], many variants are proposed to improve quality [2–4, 24, 45, 47–49, 57], increase speed [8, 9, 11, 14, 20, 32, 36, 37, 40, 43, 46, 58, 60], extend to dynamic scenes [7, 15, 18, 25, 27, 50], and so on.”