NeuMan: Neural Human Radiance Field from a Single Video

By Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan. European Conference on Computer Vision, 2022

332 citations recorded by the source corpus.

Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences. We propose a novel framework to reconstruct the human and the scene that can be rendered with novel human poses and views from just a single in-the-wild video. Given a video captured by a moving camera, we train two NeRF models: a human NeRF model and a scene NeRF model. To train these models, we rely on existing methods to estimate the rough geometry of the human and the scene. Those rough geometry estimates allow us to create a warping field from the observation space to the canonical pose-independent space, where we train the human model in. Our method is able to learn subject specific details, including cloth wrinkles and accessories, from just a 10 seconds video clip, and to provide high quality renderings of the human under novel poses, from novel views, together with the background.

View Paper: https://www.semanticscholar.org/paper/17df7e87a5d25e6e83d773e6f686eb0a85f6827e

Citations in the corpus

• NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (2022, unknown): “By separating the discrete volume rendering equation 4 piecewise, then using their newly developed AutoInt, which trains the MLP Φ θ by training its gradient (grad) networks Ψ i θ , which share internal parameters with, and are used to reassemble the integral network Φ θ .”