Lifting Object Detection Datasets into 3D

By João Carreira, Sara Alexandra Gomes Vicente, Lourdes de Agapito, Jorge P. Batista. 2015

18 citations recorded by the source corpus.

While data has certainly taken the center stage in computer vision in recent years, it can still be difficult to obtain in certain scenarios. In particular, acquiring ground truth 3D shapes of objects pictured in 2D images remains a challenging feat and this has hampered progress in recognition-based object reconstruction from a single image. Here we propose to bypass previous solutions such as 3D scanning or manual design, that scale poorly, and instead populate object category detection datasets semi-automatically with dense, per-object 3D reconstructions, bootstrapped from:(i) class labels, (ii) ground truth figure-ground segmentations and (iii) a small set of keypoint annotations. Our proposed algorithm first estimates camera viewpoint using rigid structure-from-motion and then reconstructs object shapes by optimizing over visual hull proposals guided by loose within-class shape similarity assumptions. The visual hull sampling process attempts to intersect an object's projection cone with the cones of minimal subsets of other similar objects among those pictured from certain vantage points. We show that our method is able to produce convincing per-object 3D reconstructions and to accurately estimate cameras viewpoints on one of the most challenging existing object-category detection datasets, PASCAL VOC. We hope that our results will re-stimulate interest on joint object recognition and 3D reconstruction from a single image.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

A Point Set Generation Network for 3D Object Reconstruction from a Single Image (2016, background): “Coarse correspondences in an image collection can also be used for rough 3D shape estimation [14, 3].”
Learning 3D Object Categories by Looking Around Them (2017, methodology): “Starting from datasets of unordered images, methods such as [44] and [27] use SFM and manual annotations, such as keypoints in [4, 16], to estimate a rough 3D geometry of objects.”
A Simple, Fast and Highly-Accurate Algorithm to Recover 3D Shape from 2D Landmarks on a Single Image (2016, methodology): “Example databases are the Google 3D Warehouse [16], the Carnegie Mellon Motion capture set [16], the Fine-Grained 3D Car database (FG3DCar) [3], and the PASCAL3D database [17], [18].”