AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation

By Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, Shubham Tulsiani. Computer Vision and Pattern Recognition, 2022

310 citations recorded by the source corpus.

Powerful priors allow us to perform inference with in-sufficient information. In this paper, we propose an au-toregressive prior for 3D shapes to solve multimodal 3D tasks such as shape completion, reconstruction, and gener-ation. We model the distribution over 3D shapes as a non-sequential autoregressive distribution over a discretized, low-dimensional, symbolic grid-like latent representation of 3D shapes. This enables us to represent distributions over 3D shapes conditioned on information from an arbitrary set of spatially anchored query locations and thus perform shape completion in such arbitrary settings (e.g. generating a complete chair given only a view of the back leg). We also show that the learned autoregressive prior can be leveraged for conditional tasks such as single-view reconstruction and language-based generation. This is achieved by learning task-specific ‘naive’ conditionals which can be approxi-mated by light-weight models trained on minimal paired data. We validate the effectiveness of the proposed method using both quantitative and qualitative evaluation and show that the proposed method outperforms the specialized state-of-the-art methods trained for individual tasks. The project page with code and video visualizations can be found at https://yccyenchicheng.github.io/AutoSDF/.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0e8c3f15c210909a361ba3378d6fe2822ae4f93e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, unknown): “Another approach involves structured representations (e.g., triplane [68, 7, 26] or sparse volume [63, 120, 72]) to encode 3D shapes, which better preserve spatial priors but are less efficient than vector sets.”
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation (2022, methodology): ““chair with five legs” Input AutoSDF [25]”
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization (2023, background): “Recently, there has been an increasing number of work on learning to generate a 3D implicit ﬁeld from a single image [84, 45, 66, 24, 53, 17, 20, 26, 50, 78, 49].”
LRM: Large Reconstruction Model for Single Image to 3D (2023, unknown): “Multimodal 3D Motivated by the great advances in 2D multimodal learning (Tan & Bansal, 2019; Chen et al., 2020; 2022b; Yu et al., 2022; Singh et al., 2022; Wang et al., 2022; Alayrac et al., 2022; Girdhar et al., 2023), LRM considers 3D as a new modality and directly grounds 2D feature maps onto 3D…”
LION: Latent Point Diffusion Models for 3D Shape Generation (2022, background): “[30] Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shubham Tulsiani.”
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model (2023, background): “…Gadelha et al., 2017; Smith & Meger, 2017), triangle meshes (Gao et al., 2022; Pavllo et al., 2020; Chen et al., 2019; Luo et al., 2021) or volumes (Chan et al., 2022; Or-El et al., 2022; Bergman et al., 2022; Skorokhodov et al., 2022) Diffusion models open new possibilities for 3D generation.”
3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models (2023, methodology): “The method is called Grid-8 3 because the latent grid size is 8 3 , which is exactly the same as in AutoSDF [Mittal et al. 2022].”
Point-E: A System for Generating 3D Point Clouds from Complex Prompts (2022, methodology): “Methods which train generative models directly on paired (text, 3D) data (Chen et al., 2018; Mittal et al., 2022; Fu et al., 2022; Zeng et al., 2022) or unlabeled 3D data (Sanghi et al., 2021; 2022; Watson et al., 2022).”