DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation

By Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, Gang Zeng. International Conference on Learning Representations, 2023

1,030 citations recorded by the source corpus.

Recent advances in 3D content creation mostly leverage optimization-based 3D generation via score distillation sampling (SDS). Though promising results have been exhibited, these methods often suffer from slow per-sample optimization, limiting their practical usage. In this paper, we propose DreamGaussian, a novel 3D content generation framework that achieves both efficiency and quality simultaneously. Our key insight is to design a generative 3D Gaussian Splatting model with companioned mesh extraction and texture refinement in UV space. In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significantly faster for 3D generative tasks. To further enhance the texture quality and facilitate downstream applications, we introduce an efficient algorithm to convert 3D Gaussians into textured meshes and apply a fine-tuning stage to refine the details. Extensive experiments demonstrate the superior efficiency and competitive generation quality of our proposed approach. Notably, DreamGaussian produces high-quality textured meshes in just 2 minutes from a single-view image, achieving approximately 10 times acceleration compared to existing methods.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cc1a674bb164d09a060cf5b26fe518c02fae0ddc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

LangSplat: 3D Language Gaussian Splatting (2023, background): “[38] introduced [48], super-resolution [25], image matting [45], object tracking [11, 42], medical image segmentation [29], image editing [12], and so on.”
Text-to-3D using Gaussian Splatting (2023, methodology): “Our concurrent work DreamGaussian [70] achieves fast image-to-3D by capitalizing on the rapid convergence of Gaussian Splatting, whose contribution is orthogonal to ours since we focus on incorporating 3D prior with more advanced representation.”
Gaussian Splatting SLAM (2023, positive): “Several works have applied 3D Gaussians and differentiable rendering to static scene capture [11, 38], and in particular more recent works utilise 3DGS and demonstrate superior results in vision tasks such as dynamic scene capture [15, 42, 44] and 3D generation [35, 45].”
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching (2023, background): “For example, some of them [5, 6, 12, 21, 29, 43, 47] improve the visual quality of text-to-3D distillation via modifying NeRF or introducing other advanced 3D representations.”
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, methodology): “The baselines include SOTA single-image 3D generation methods: ShapE [36], One-2-3-45 [54], One-2-3-45++ [53], DreamGaus-sian [93], Wonder3D [58], TriplaneGaussian [114], and LGM [92].”
CAT3D: Create Anything in 3D with Multi-View Diffusion Models (2024, background): “Research in this space has aimed to improve distillation strategies [18, 19, 20, 21, 22], swap in other 3D representations [23, 24, 25, 26, 27], and amortize the optimization process [28].”
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, methodology): “We compare our SV3D-guided 3D generations with several prior methods including Point-E [33], Shap-E [16], One-2-3-45++ [23], Dream-Gaussian [49], SyncDreamer [26], EscherNet [20], Free3D [62], and Stable Zero123 [48].”
PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (2024, neutral): “N OVEL view synthesis and geometry reconstruction are challenging and crucial tasks in computer vision, widely used in AR/VR [1], [2], 3D content generation [3], [4], [5], [6], and autonomous driving.”
Gaussian Grouping: Segment and Edit Anything in 3D Scenes (2023, background): “Another stream of works [6, 43, 54] focus on the 3D content creation, which combine Gaussian Splatting with the diffusion-based models and show high-quality generation results.”
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting (2023, neutral): “Moreover, it has been observed in [7, 45, 53] that optimizing Gaussian Splatting (GS) using highly random generative guidance like Score Distillation Sampling [36] poses signiﬁcant challenges.”
PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (2023, background): “However, when the object undergoes rotations, the spherical harmonic bases are still represented in the material space, resulting in varying appearances even if the view direction is fixed relatively to the object.”
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (2023, methodology): “Within the optimization-based approaches, our baselines include DreamFusion [49] with Zero123 XL [34] as its backbone, as well as SyncDreamer [36], and DreamGaussian [63].”
VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (2024, neutral): “It is also applied to dynamic scene reconstruction [28, 51, 55, 56] and 3D content generation [12, 42, 59].”
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (2024, methodology): “While this method excels with dense input views and has achieved success in various 3D tasks [23, 39, 47], its reconstruction with sparse view inputs remains an open problem.”
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, positive): “Initially, score distillation was adopted to harness the generation power of image diffusion models for 3D content (texture) synthesis [87, 51, 62, 70].”
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation (2024, methodology): “We then compare with recent methods [16,47] on text-to-3D tasks.”
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (2023, methodology): “Here come two main streams for achieving this goal: (i) training a new diffusion model with 3D data [12, 15, 24, 48] (namely the 3D diffusion model ) and (ii) lifting the 2D diffusion model to 3D [2, 6, 9, 32, 52, 58, 67, 70, 76, 77, 81, 90].”