One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

By Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, Hao Su. Computer Vision and Pattern Recognition, 2023

349 citations recorded by the source corpus.

Recent advancements in open-world 3D object generation have been remarkable, with image-to-3D methods of-fering superior fine-grained control over their text-to-3D counterparts. However, most existing models fall short in simultaneously providing rapid generation speeds and high fidelity to input images - two features essential for practi-cal applications. In this paper, we present One-2-3-45++, an innovative method that transforms a single image into a detailed 3D textured mesh in approximately one minute. Our approach aims to fully harness the extensive knowledge embedded in 2D diffusion models and priors from valuable yet limited 3D data. This is achieved by initially finetuning a 2D diffusion model for consistent multi-view image generation, followed by elevating these images to 3D with the aid of multi-view-conditioned 3D native diffusion models. Extensive experimental evaluations demonstrate that our method can produce high-quality, diverse 3D assets that closely mirror the original input image.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/05782b157d5468c08de9dbde9635ee9902f40b60

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Jiajun Wu
2. Gordon Wetzstein
3. Varun Jampani
4. Yinghao Xu
5. Hao Su
6. Robin Rombach
7. Sergey Levine
8. Sida Peng

In Computer Graphics and Computer-Aided Design

1. Ying Shan
2. Xintao Wang
3. Gordon Wetzstein
4. Weihao Cheng
5. Yinghao Xu
6. Hao Su
7. Robin Rombach
8. Sergey Levine

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Chen Change Loy
3. Ming-Hsuan Yang
4. Dacheng Tao
5. Trevor Darrell
6. Xiaogang Wang
7. Andrea Vedaldi
8. Jitendra Malik

Citations in the corpus

• Evaluating Real-World Robot Manipulation Policies in Simulation (2024, positive): “…public repositories (e.g., Objaverse [15]), from 3D scanning of objects purchased from Amazon, from single-view 3D generation (e.g., One-2-3-45++ [39]), or from manual modeling based on precise measurements of real-world counterparts (we only used the last technique for articulated objects like…”
• InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models (2024, methodology): “With 3D consistent multi-view images, various techniques can be applied to obtain the 3D object, e.g ., SDS optimization [51], neural surface reconstruction meth-2 ods [24, 26], multi-view-conditioned 3D diffusion models [21].”
• GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation (2024, background): “LGM [92], One-2-3-45 [54], One-2-3-45++ [53], and Wonder3D [58] also combine multiview diffusion and reconstruction for single image-to-3D generation, but they produce blurrier texture and noticeable geometry artifacts.”
• SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, methodology): “One-2-3-45 [24] and One-2-3-45++ [23] train additional 3D network using the outputs of 2D generator.”