ShapeFormer: Transformer-based Shape Completion via Sparse Representation

By Xingguang Yan, Liqiang Lin, Niloy J. Mitra, Dani Lischinski, Daniel Cohen-Or, Hui Huang. 2022

167 citations recorded by the source corpus.

We present ShapeFormer, a transformer-based network that produces a distribution of object completions, conditioned on incomplete, and possibly noisy, point clouds. The resultant distribution can then be sampled to generate likely completions, each exhibiting plausible shape details while being faithful to the input. To facilitate the use of transformers for 3D, we introduce a compact 3D representation, vector quantized deep implicit function (VQDIF), that utilizes spatial sparsity to represent a close approximation of a 3D shape by a short sequence of discrete variables. Experiments demonstrate that ShapeFormer outperforms prior art for shape completion from ambiguous partial inputs in terms of both completion quality and diversity. We also show that our approach effectively handles a variety of shape types, incomplete patterns, and real-world scans.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Model Architectures, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Justin Johnson
2. Matthias Nießner
3. Georgia Gkioxari
4. Jitendra Malik
5. Shubham Tulsiani
6. Peter Wonka
7. Jiapeng Tang
8. Biao Zhang

In Computer Graphics and Computer-Aided Design

1. Matthias Nießner
2. Justin Johnson
3. Jitendra Malik
4. Georgia Gkioxari
5. Shubham Tulsiani
6. Peter Wonka
7. Jiapeng Tang
8. Biao Zhang

In Model Architectures

1. Christoph Feichtenhofer
2. Jitendra Malik
3. Justin Johnson
4. Shubham Tulsiani
5. Peter Wonka
6. Georgia Gkioxari
7. Maneesh Singh
8. Matthias Nießner

In 3D Vision & Pose

1. Jitendra Malik
2. Justin Johnson
3. Georgia Gkioxari
4. Shubham Tulsiani
5. Matthias Nießner
6. Christoph Feichtenhofer
7. Chaozheng Wu
8. Peter Wonka

Citations in the corpus

• Multiview Compressive Coding for 3D Reconstruction (2023, methodology): “For objects, methods directly output full point clouds [21,79,80] or deploy generative models [76, 84], but are typically tied to a fixed resolution.”
• AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation (2022, methodology): “[44] proposes VQDIF to obtain the sparse representation of the partial point cloud, and adopts ShapeFormer to output the complete shapes.”
• 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models (2023, methodology): “A lot of recent work employs ARs [Cheng et al. 2022; Mittal et al. 2022; Nash et al. 2020; Sun et al. 2020; Yan et al. 2022; Zhang et al. 2022].”