Self-supervised Learning of 3D Objects from Natural Images

By Hiroharu Kato, Tatsuya Harada. 2019

7 citations recorded by the source corpus.

We present a method to learn single-view reconstruction of the 3D shape, pose, and texture of objects from categorized natural images in a self-supervised manner. Since this is a severely ill-posed problem, carefully designing a training method and introducing constraints are essential. To avoid the difficulty of training all elements at the same time, we propose training category-specific base shapes with fixed pose distribution and simple textures first, and subsequently training poses and textures using the obtained shapes. Another difficulty is that shapes and backgrounds sometimes become excessively complicated to mistakenly reconstruct textures on object surfaces. To suppress it, we propose using strong regularization and constraints on object surfaces and background images. With these two techniques, we demonstrate that we can use natural image collections such as CIFAR-10 and PASCAL objects for training, which indicates the possibility to realize 3D object reconstruction on diverse object categories beyond synthetic datasets.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

In Representation & Self-Supervised Learning

Citations in the corpus

GAN Inversion: A Survey (2021, background): “view synthesis [222], [222], face shape generation [223], object modeling [224], [225], and human reconstruction [226], [227], [228], [229].”