GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

By Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, K. Yin, Daiqing Li, Or Litany, Zan Gojcic, Sanja Fidler. Neural Information Processing Systems, 2022

636 citations recorded by the source corpus.

As several industries are moving towards modeling massive 3D virtual worlds, the need for content creation tools that can scale in terms of the quantity, quality, and diversity of 3D content is becoming evident. In our work, we aim to train performant 3D generative models that synthesize textured meshes which can be directly consumed by 3D rendering engines, thus immediately usable in downstream applications. Prior works on 3D generative modeling either lack geometric details, are limited in the mesh topology they can produce, typically do not support textures, or utilize neural renderers in the synthesis process, which makes their use in common 3D software non-trivial. In this work, we introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. We bridge recent success in the differentiable surface modeling, differentiable rendering as well as 2D Generative Adversarial Networks to train our model from 2D image collections. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings, achieving significant improvements over previous methods.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/a57d47b762341340656d5b5caa84f370d9d31063

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Control and Systems Engineering

Citations in the corpus

ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation (2023, methodology): “The evolution of these techniques, from Generative Adversarial Networks (GANs)[3, 9, 10, 24, 25, 27] to diffusion-based frameworks[33], marks a notable progression.”
Generative artificial intelligence (2023, positive): “3D modeling is also impacted by GAI applications such as DreamFusion (Poole et al., 2023), Nvidia GET3D (Gao et al., 2022), and Point-E (Nichol et al., 2022), which generate realistic and complex 3D models that facilitate a range of applications from product design and architecture to virtual realit”
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, positive): “Leveraging various successful 2D GAN architectures [4, 23, 37, 39–42, 112], 3D GANs [5, 6, 22 , 24 , 65 , 67 , 77 , 82 , 87 , 88 , Some concurrent works, such as LGM [92], AGG [103], and Splatter Image [90], also use 3D Gaussians in a feed-forward model.”
RODIN: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion (2022, background): “3D-aware GANs [9, 10, 12, 16, 18, 41, 54–58, 62, 71, 77] recently attract significant research interest, which are trained to produce radiance fields with image level distribution matching.”
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation (2023, methodology): “In this section, we adopt a state-of-the-art generative model that directly generates explicit textured 3D meshes, namely GET3D [22].”
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model (2023, methodology): “…different methods to generate 3D models in the form of point clouds (Wu et al., 2016; Gadelha et al., 2017; Smith & Meger, 2017), triangle meshes (Gao et al., 2022; Pavllo et al., 2020; Chen et al., 2019; Luo et al., 2021) or volumes (Chan et al., 2022; Or-El et al., 2022; Bergman et al., 2022;…”
Zero-1-to-3: Zero-shot One Image to 3D Object (2023, background): “Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. in the direction of open-world 3D reconstruction by pre-training on large-scale, diverse datasets such as CO3D [44, 31, 36, 16].”
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, neutral): “Another approach involves structured representations (e.g., triplane [68, 7, 26] or sparse volume [63, 120, 72]) to encode 3D shapes, which better preserve spatial priors but are less efficient than vector sets.”
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (2023, methodology): “Here come two main streams for achieving this goal: (i) training a new diffusion model with 3D data [12, 15, 24, 48] (namely the 3D diffusion model ) and (ii) lifting the 2D diffusion model to 3D [2, 6, 9, 32, 52, 58, 67, 70, 76, 77, 81, 90].”
MVDream: Multi-view Diffusion for 3D Generation (2023, background): “With Generative Adversarial Networks (GANs) yielding improved results in image synthesis, numerous studies have investigated 3D-aware GANs (Nguyen-Phuoc et al., 2019; 2020; Niemeyer & Geiger, 2021; Deng et al., 2022; Chan et al., 2022; Gao et al., 2022).”
Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers (2023, neutral): “They aim to exploit techniques such as GAN [16], diffusion model [8, 27, 41, 78, 81] to sample a high 3D content from pre-trained distribution.”
LRM: Large Reconstruction Model for Single Image to 3D (2023, neutral): “…3D Motivated by the great advances in 2D multimodal learning (Tan & Bansal, 2019; Chen et al., 2020; 2022b; Yu et al., 2022; Singh et al., 2022; Wang et al., 2022; Alayrac et al., 2022; Girdhar et al., 2023), LRM considers 3D as a new modality and directly grounds 2D feature maps onto 3D…”
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image (2023, background): “Recent NeRF-GAN methods [4, 15, 22, 24, 47, 58] learn to generate NeRFs for specific categories like human or cat faces.”
Point-E: A System for Generating 3D Point Clouds from Complex Prompts (2022, background): “More recently, Gao et al. (2022) train a GAN that directly produces full 3D meshes, paired with a discriminator that inputs differentiably-rendered (Laine et al., 2020) views of the generated meshes.”
Wonder3D: Single Image to 3D Using Cross-Domain Diffusion (2023, background): “There exists another group of works that endeavor to directly produce 3D geometries like point clouds [41, 45, 75, 80], meshes [16, 37], neural fields [1, 4, 8, 15, 17, 21, 26– 28, 44, 46, 65, 76] via network inference to avoid time-consuming per-shape optimization.”
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers (2023, methodology): “We additionally compare with a state-of-the-art neural field-based method, GET3D [14], that creates shapes as 3D signed distance fields (SDFs) from which a mesh is extracted by differentiable marching tetrahedra.”
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures (2022, methodology): “For example, in GET3D [18], two generators are trained, one generates a 3D SDF, and the other a texture ﬁeld.”
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (2023, background): “Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. els [8, 16, 37, 87], a majority rely solely on 3D shape datasets for training.”