Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image

By Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker. 2019

329 citations recorded by the source corpus.

We propose a deep inverse rendering framework for indoor scenes. From a single RGB image of an arbitrary indoor scene, we obtain a complete scene reconstruction, estimating shape, spatially-varying lighting, and spatially-varying, non-Lambertian surface reflectance. Our novel inverse rendering network incorporates physical insights -- including a spatially-varying spherical Gaussian lighting representation, a differentiable rendering layer to model scene appearance, a cascade structure to iteratively refine the predictions and a bilateral solver for refinement -- allowing us to jointly reason about shape, lighting, and reflectance. Since no existing dataset provides ground truth high quality spatially-varying material and spatially-varying lighting, we propose novel methods to map complex materials to existing indoor scene datasets and a new physically-based GPU renderer to create a large-scale, photorealistic indoor dataset. Experiments show that our framework outperforms previous methods and enables various novel applications like photorealistic object insertion and material editing.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

  1. Jonathan T. Barron
  2. Andreas Geiger
  3. Ben Mildenhall
  4. Pratul P. Srinivasan
  5. Matthew Tancik
  6. Angjoo Kanazawa
  7. Zexiang Xu
  8. Kalyan Sunkavalli

In Computer Graphics and Computer-Aided Design

  1. Jonathan T. Barron
  2. Ben Mildenhall
  3. Pratul P. Srinivasan
  4. Andreas Geiger
  5. Matthew Tancik
  6. Zexiang Xu
  7. Angjoo Kanazawa
  8. Kalyan Sunkavalli

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Andreas Geiger
  3. Jitendra Malik
  4. Andrea Vedaldi
  5. Ming-Hsuan Yang
  6. Dacheng Tao
  7. Trevor Darrell
  8. Xiaogang Wang

In 3D Vision & Pose

  1. Jonathan T. Barron
  2. Zexiang Xu
  3. Matthew Tancik
  4. Angjoo Kanazawa
  5. Ben Mildenhall
  6. Kalyan Sunkavalli
  7. Pratul P. Srinivasan
  8. Noah Snavely

Citations in the corpus