LRM: Large Reconstruction Model for Single Image to 3D

By Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, Hao Tan. International Conference on Learning Representations, 2023

922 citations recorded by the source corpus.

We propose the first Large Reconstruction Model (LRM) that predicts the 3D model of an object from a single input image within just 5 seconds. In contrast to many previous methods that are trained on small-scale datasets such as ShapeNet in a category-specific fashion, LRM adopts a highly scalable transformer-based architecture with 500 million learnable parameters to directly predict a neural radiance field (NeRF) from the input image. We train our model in an end-to-end manner on massive multi-view data containing around 1 million objects, including both synthetic renderings from Objaverse and real captures from MVImgNet. This combination of a high-capacity model and large-scale training data empowers our model to be highly generalizable and produce high-quality 3D reconstructions from various testing inputs, including real-world in-the-wild captures and images created by generative models. Video demos and interactable 3D meshes can be found on our LRM project webpage: https://yiconghong.me/LRM.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/eb2cbd12f749f14716296f7f415e921562c9079b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis (2024, methodology): “LRM [26] extends this strategy to generic objects, while other methods like [7], [8], [9] adopt the multi-plane representation, and PixelNeRF [10] employs NeRF [1] as 3D representation for novel view synthesis.”
Splatter Image: Ultra-Fast Single-View 3D Reconstruction (2023, methodology): “4 shows that our method produces sharper and more accurate reconstructions than prior works in challenging situations with limited visibility and thin structures.”
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, methodology): “GRM implements two key insights: first, we replace the triplane scene representation of recent feed-forward generative frameworks [30,46,106] by 3D Gaussians; second, we design a pure transformer architecture to translate the set of input pixels to the set of pixel-aligned 3D Gaussians defining the…”
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, neutral): “This strategy is effective for decoupling the difficulties of shape and texture generation [34, 107, 46, 47] and also provides flexibility for texturing either generated or handcrafted meshes.”
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting (2024, methodology): “We empirically find that the perceptual loss in [14] based on VGG-19 network [55] provides a more stable training than LPIPS [75] used in [27,32,63,68], and we use it in this work.”
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, background): “Some works [6, 15, 26] try to address the multi-view consistency by jointly reasoning a 3D representation during NVS, but this comes at the cost of high computational and data requirements, often still resulting in unsatisfactory re-sults with inconsistent geometric and texture details.”
VGGT: Visual Geometry Grounded Transformer (2025, positive): “There are several recent examples of large 3D neural networks, including DepthAnything [97], MoGe [86], and LRM [34].”
InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models (2024, methodology): “(i) Following LRM [14] and Instant3D [19], our transformer-based triplane decoder produces 64 × 64 triplanes, whose resolution may be a bottleneck for high-definition 3D modeling.”
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation (2024, methodology): “For instance, given a fixed compute budget, the triplane representation of LRM [15] is limited to the resolution of 32 , while the resolution of the rendered image is capped at 128 due to the online volume rendering.”
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model (2023, methodology): “In the second stage, instead of relying on a slow optimization-based reconstruction method, inspired by (Hong et al., 2023), we introduce a novel sparse-view large reconstruction model with a transformer-based architecture that can directly regress a triplane-based (Chan et al., 2022) NeRF from a…”
Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers (2023, positive): “In order to better utilize camera information in our pipeline, we implement an adaptive layer norm (adaLN) to modulate the DINOv2 features with the camera features, which is similar to [21, 48].”
CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets (2024, positive): “For example, Instant3D [Li et al. 2023], LRM [Hong et al. 2024; Wang et al. 2024], DMV3D [Xu et al. 2024] and TGS [Zou et al. 2024] further utilized sparse-view or single-view reconstructors that leverage a Vision Transformer (ViT) as the vision backbone, coupled with a deep transformer…”
Depth Anything 3: Recovering the Visual Space from Any Views (2025, positive): “Early methods adopted NeRF as the underlying 3D representation [12, 14, 35, 52, 107, 119], but recent work has largely shifted to 3DGS due to its explicit structure and real-time rendering.”
CAT3D: Create Anything in 3D with Multi-View Diffusion Models (2024, methodology): “…regularization techniques targeted for sparse-view reconstruction [4, 5], feed-forward models trained to create 3D objects from single images [6], or the use of image-conditioned [7] or text-conditioned [8] generative priors in the optimization process—but each of these specialized methods…”