LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation

By Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, Ziwei Liu. European Conference on Computer Vision, 2024

817 citations recorded by the source corpus.

3D content creation has achieved significant progress in terms of both quality and speed. Although current feed-forward models can produce 3D objects in seconds, their resolution is constrained by the intensive computation required during training. In this paper, we introduce Large Multi-View Gaussian Model (LGM), a novel framework designed to generate high-resolution 3D models from text prompts or single-view images. Our key insights are two-fold: 1) 3D Representation: We propose multi-view Gaussian features as an efficient yet powerful representation, which can then be fused together for differentiable rendering. 2) 3D Backbone: We present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models. Extensive experiments demonstrate the high fidelity and efficiency of our approach. Notably, we maintain the fast speed to generate 3D objects within 5 seconds while boosting the training resolution to 512, thereby achieving high-resolution 3D content generation.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/11665dbecb17ef4d3d71b75b8666ce0e61bd43fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting (2024, methodology): “In addition, we compare with the concurrent work LGM [61] which is also based on Gaussian Splatting [30] .”
A Survey on 3D Gaussian Splatting (2024, neutral): “The integration of GS into robotic systems has yielded advancements across three core domains.”
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, methodology): “The baselines include SOTA single-image 3D generation methods: ShapE [36], One-2-3-45 [54], One-2-3-45++ [53], DreamGaus-sian [93], Wonder3D [58], TriplaneGaussian [114], and LGM [92].”
Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (2024, background): “…rapidly expanded 3D-GS into diverse downstream applications beyond static 3D reconstruction, sparking a surge of extended applications to 3D generative modeling [31]–[33], physical simulation [13], [34], dynamic modeling [35]–[37], SLAMs [38], [39], and autonomous driving scenes [10]–[12], etc.”
Structured 3D Latents for Scalable and Versatile 3D Generation (2024, neutral): “IV.”
Gen3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (2025, neutral): “Alternative approaches seek to train a feed-forward model to predict novel views from sparse posed images [6–8, 22, 42, 51, 55, 67, 73], but these methods are limited by the scarcity of training data and struggle to generalize to unseen domains and extreme novel views.”
Recent advances in 3D Gaussian splatting (2024, neutral): “The U-Net is asymmetric, which allows for the input of high-resolution images while limiting the number of output Gaussians.”
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images (2024, methodology): “Inspired by existing multi-view based models [32, 38], we also flatten the features before applying self-attention, allowing information to propagate among different cost volumes.”
InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models (2024, methodology): “Different from view-space reconstruction models [13, 45, 46, 63], our model reconstruct 3D objects in a canonical world space where the z -axis aligns with the anti-gravity direction.”