Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision

By Michael Niemeyer, Lars M. Mescheder, Michael Oechsle, Andreas Geiger. 2019

1,118 citations recorded by the source corpus.

Learning-based 3D reconstruction methods have shown impressive results. However, most methods require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differentiable rendering techniques to train reconstruction models from RGB images. Unfortunately, these approaches are currently restricted to voxel- and mesh-based representations, suffering from discretization or low resolution. In this work, we propose a differentiable rendering formulation for implicit shape and texture representations. Implicit representations have recently gained popularity as they represent shape and texture continuously. Our key insight is that depth gradients can be derived analytically using the concept of implicit differentiation. This allows us to learn implicit shape and texture representations directly from RGB images. We experimentally show that our single-view reconstructions rival those learned with full 3D supervision. Moreover, we find that our method can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

  1. Andreas Geiger
  2. Jonathan T. Barron
  3. Michael Niemeyer
  4. Gerard Pons-Moll
  5. Matthias Nießner
  6. Ben Mildenhall
  7. Songyou Peng
  8. Pratul P. Srinivasan

In Computer Graphics and Computer-Aided Design

  1. Andreas Geiger
  2. Jonathan T. Barron
  3. Ben Mildenhall
  4. Pratul P. Srinivasan
  5. Matthias Nießner
  6. Matthew Tancik
  7. Peter Hedman
  8. Michael Niemeyer

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Jitendra Malik
  3. Ming-Hsuan Yang
  4. Dacheng Tao
  5. Andrea Vedaldi
  6. Trevor Darrell
  7. Xiaogang Wang
  8. Andrew Zisserman

In 3D Vision & Pose

  1. Andreas Geiger
  2. Michael Niemeyer
  3. Jonathan T. Barron
  4. Matthias Nießner
  5. Jitendra Malik
  6. Justin Johnson
  7. Matthew Tancik
  8. Andrea Vedaldi

Citations in the corpus