Semantically-enriched 3D models for common-sense knowledge

By Manolis Savva, Angel X. Chang, P. Hanrahan. 2015

129 citations recorded by the source corpus.

We identify and connect a set of physical properties to 3D models to create a richly-annotated 3D model dataset with data on physical sizes, static support, attachment surfaces, material compositions, and weights. To collect these physical property priors, we leverage observations of 3D models within 3D scenes and information from images and text. By augmenting 3D models with these properties we create a semantically rich, multi-layered dataset of common indoor objects. We demonstrate the usefulness of these annotations for improving 3D scene synthesis systems, enabling faceted semantic queries into 3D model datasets, and reasoning about how objects can be manipulated by people using weight and static friction estimates.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology, 3D Vision & Pose, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

In 3D Vision & Pose

In Knowledge Graphs & Representation

Citations in the corpus

IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation (2019, methodology): “Scene Flow [15] has over 39,000 pairs to train the network, which is based on the 3D model provided by ShapeNet [25] and the texture from Flicker.”
ShapeNet: An Information-Rich 3D Model Repository (2015, background): “More details about the acquisition of these physical attribute annotations are available separately [26].”
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016, background): “[18] can be seen as a three-dimensional version of Chairs: 22k renderings of random scenes show 3D models from the ShapeNet dataset [23] moving in front of static 3D backgrounds.”
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation (2015, methodology): “To populate the scene, we downloaded 35927 detailed 3D models from Stanford’s ShapeNet6 [18] database.”
Multi-view 3D Models from Single Images with a Convolutional Network (2015, methodology): “We used synthetic data from the ShapeNet dataset [1] for training the networks.”
Language Grounding with 3D Objects (2021, methodology): “To construct SNARE, we select a subset of 7 , 897 ACRONYM [44] object models from ShapeNet-Sem [45, 13].”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “In contrast to FlowNet and SpyNet, they use the FlyingThings3D dataset (Mayer et al. (2016)) consisting of 22k renderings of static 3D scenes with moving 3D models from ShapeNet dataset (Savva et al. (2015)).”
InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset (2018, background): “Although an excellent resource, the quality of the models and textures from these repositories can be quite variable and missing ground truth metric scales requires automated prediction [26, 27].”
Learning Semantic Deformation Flows with 3D Convolutional Networks (2016, methodology): “3D ShapeNets [40] introduced 3D deep learning for modeling shapes as volumetrically discretized (i.e., in voxel form) data, and showed that intuitive 3D features can be learned directly in 3D. Song et al. [37] introduced an amodal 3D object detector method for RGB-D images using a two 3D convolution”
Scaling Out-of-Distribution Detection for Real-World Settings (2022, methodology): “We insert anomalies taken from the Digimation Model Bank Library and semantic ShapeNet (ShapeNetSem) (Savva et al., 2015) into the test set in order to evaluate methods for out-of-distribution detection.”