Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation

By Edward James Smith, Scott Fujimoto, David Meger. 2018

56 citations recorded by the source corpus.

We consider the problem of scaling deep generative shape models to high-resolution. Drawing motivation from the canonical view representation of objects, we introduce a novel method for the fast up-sampling of 3D objects in voxel space through networks that perform super-resolution on the six orthographic depth projections. This allows us to generate high-resolution objects with more efficient scaling than methods which work directly in 3D. We decompose the problem of 2D depth super-resolution into silhouette and depth prediction to capture both structure and fine detail. This allows our method to generate sharp edges more easily than an individual network. We evaluate our work on multiple experiments concerning high-resolution 3D objects, and show our system is capable of accurately predicting novel objects at resolutions as large as 512$\mathbf{\times}$512$\mathbf{\times}$512 -- the highest resolution reported for this task. We achieve state-of-the-art performance on 3D object reconstruction from RGB images on the ShapeNet dataset, and further demonstrate the first effective 3D super-resolution method.

Subfields: Computer Vision and Pattern Recognition, Media Technology, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In 3D Vision & Pose

Citations in the corpus

Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, methodology): “As a ﬁnal conditional task, we apply occupancy networks to 3D super-resolution [62].”
Mesh R-CNN (2019, methodology): “Evaluation We adopt evaluation metrics used in recent work [56, 57, 69].”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, methodology): “Higher resolutions have been used [74, 73, 81] at the cost of limited training batches and slow training or lossy 2D projections [63].”
A Deep Journey into Super-resolution (2019, methodology): “For SR in higher dimensional inputs (such as videos and 3D scans), we refer the reader to recent seminal works [1], [2], [3], [4], [5].”
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, background): “Smith et al. [93] ﬁrst estimate a low resolution voxel grid.”