pixelNeRF: Neural Radiance Fields from One or Few Images

By Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa. 2020

2,248 citations recorded by the source corpus.

We propose pixelNeRF, a learning framework that predicts a continuous neural scene representation conditioned on one or few input images. The existing approach for constructing neural radiance fields [27] involves optimizing the representation to every scene independently, requiring many calibrated views and significant compute time. We take a step towards resolving these shortcomings by introducing an architecture that conditions a NeRF on image inputs in a fully convolutional manner. This allows the network to be trained across multiple scenes to learn a scene prior, enabling it to perform novel view synthesis in a feed-forward manner from a sparse set of views (as few as one). Leveraging the volume rendering approach of NeRF, our model can be trained directly from images with no explicit 3D supervision. We conduct extensive experiments on ShapeNet benchmarks for single image novel view synthesis tasks with held-out objects as well as entire unseen categories. We further demonstrate the flexibility of pixelNeRF by demonstrating it on multi-object ShapeNet scenes and real scenes from the DTU dataset. In all cases, pixelNeRF outperforms current state-of-the-art baselines for novel view synthesis and single image 3D reconstruction. For the video and code, please visit the project website:https://alexyu.net/pixelnerf.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, 3D Vision & Pose, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

  1. Andreas Geiger
  2. Matthias Nießner
  3. Michael Niemeyer
  4. Jonathan T. Barron
  5. Justin Johnson
  6. Andrea Tagliasacchi
  7. Songyou Peng
  8. Ben Mildenhall

In Computer Graphics and Computer-Aided Design

  1. Andreas Geiger
  2. Jonathan T. Barron
  3. Ben Mildenhall
  4. Pratul P. Srinivasan
  5. Matthias Nießner
  6. Michael Niemeyer
  7. Matthew Tancik
  8. Vincent Sitzmann

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Jitendra Malik
  3. Dacheng Tao
  4. Andrea Vedaldi
  5. Ming-Hsuan Yang
  6. Trevor Darrell
  7. Xiaogang Wang
  8. Andrew Zisserman

In 3D Vision & Pose

  1. Andreas Geiger
  2. Matthias Nießner
  3. Michael Niemeyer
  4. Jonathan T. Barron
  5. Jitendra Malik
  6. Justin Johnson
  7. Angela Dai
  8. Andrea Vedaldi

In Meta & Continual Learning

  1. Zexiang Xu
  2. Andreas Geiger
  3. Andrea Tagliasacchi
  4. Deva Ramanan
  5. Jonathan T. Barron
  6. Andrea Vedaldi
  7. Marco Pavone
  8. Jiawei Yang

Citations in the corpus