Learning Shape Abstractions by Assembling Volumetric Primitives

By Shubham Tulsiani, Hao Su, Leonidas J. Guibas, Alexei A. Efros, Jitendra Malik. 2016

404 citations recorded by the source corpus.

We present a learning framework for abstracting complex shapes by learning to assemble objects using 3D volumetric primitives. In addition to generating simple and geometrically interpretable explanations of 3D objects, our framework also allows us to automatically discover and exploit consistent structure in the data. We demonstrate that using our method allows predicting shape representations which can be leveraged for obtaining a consistent parsing across the instances of a shape collection and constructing an interpretable shape similarity measure. We also examine applications for image-based prediction as well as shape manipulation.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

A Papier-Mache Approach to Learning 3D Surface Generation (2018, positive): “…work deforms primitive surface elements into a 3D shape, it can be seen as bridging the gap between the recent works that learn to represent 3D shapes as a set of simple primitives, with a ﬁxed, low number of parameters [31] and those that represent 3D shapes as an unstructured set of points [8].”
Hierarchical Surface Prediction for 3D Object Reconstruction (2017, background): “It has also been shown that CNNs can be leveraged to predict alternative primitive based representations [33].”
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs (2017, neutral): “[33] assemble objects from volumetric primitives.”
Learning Gradient Fields for Shape Generation (2020, neutral): “They also allow for learning a structured decomposition of shapes, representing local regions with Gaussian functions [21,22] or other primitives [58,53,28].”
BSP-Net: Generating Compact Meshes via Binary Space Partitioning (2019, methodology): “For 3D shape autoencoding, we compare BSP-Net to a few other shape decomposition networks: Volumetric Primitives (VP) [43], Super Quadrics (SQ) [32], and Branched Auto Encoders (BAE) [4].”
Local Deep Implicit Functions for 3D Shape (2019, methodology): “Learned representations include SDM-Net [12], GRASS [20], CSGNet [35], Volumetric Primitives [39], Superquadrics [28], and SIF [13].”
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images Using a View-Based Representation (2020, background): “…has been studied extensively in the computer vision andgraphics communities (Saxena et al. 2009; Chaudhuri et al. 2011; Kalogerakis et al. 2012; Chang et al. 2015; Rezende et al. 2016; Soltani et al. 2017; Kulkarni et al. 2015; Tulsiani et al. 2016; Huang et al. 2019; Jiang et al. 2019).”
What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction (2021, background): “2 [51] Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A”
Learning Shape Templates With Structured Implicit Functions (2019, methodology): “Tulsiani et al. [51] proposed a neural network that learned placements for a small number (3 to 6) of box primitives from image or shape inputs, without additional supervision.”
Hierarchical Relational Inference (2020, methodology): “In this case, hierarchies are either learned from 3D volumetric primitives (Tulsiani et al. 2017), or using ground truth pixel-wise flow fields to guide object or object parts segmentation (Liu et al. 2019).”
Tangent Convolutions for Dense Prediction in 3D (2018, background): “[54] approximate 3D models with volumetric primitives in an end-to-end differentiable framework, and use this representation for solving several tasks.”
3D-Aware Scene Manipulation via Inverse Graphics (2018, background): “Many recent papers have explored the problem of 3D reconstruction from a single color image, depth map, or silhouette [Choy et al., 2016, Kar et al., 2015, Tatarchenko et al., 2016, Tulsiani et al., 2017, Wu et al., 2017b, 2016b, Yan et al., 2016b, Soltani et al., 2017].”
Learning to Infer and Execute 3D Shape Programs (2019, methodology): “A few recent papers have studied modeling 3D shapes as a collection of primitives (Tulsiani et al., 2017), with simple operations such as addition and subtraction (Sharma et al.”
Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling (2019, methodology): “Additionally, [28,32] provide datasets with structured representations of single objects.”
Im2Vec: Synthesizing Vector Graphics without Vector Supervision (2021, background): “Tulsiani et al [39] also use hierarchies of cuboids, but from occupancy volumes.”
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression (2017, methodology): “The work of [26] decomposes an input 3D shape into shape primitives which along with a set of parameters can be used to re-assemble the given shape.”