OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation

By Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Chen Qian, Dahua Lin and 1 more. Computer Vision and Pattern Recognition, 2023

432 citations recorded by the source corpus.

Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale real-scanned 3D databases. To facilitate the development of 3D perception, reconstruction, and generation in the real world, we propose OmniObject3D, a large vocabulary 3D object dataset with massive high-quality real-scanned 3D objects. OmniObject3D has several appealing properties: 1) Large Vocabulary: It comprises 6,000 scanned objects in 190 daily categories, sharing common classes with popular 2D datasets (e.g., ImageNet and LVIS), benefiting the pursuit of generalizable 3D representations. 2) Rich Annotations: Each 3D object is captured with both 2D and 3D sensors, providing textured meshes, point clouds, multi-view rendered images, and multiple real-captured videos. 3) Realistic Scans: The professional scanners support high-quality object scans with precise shapes and realistic appearances. With the vast exploration space offered by OmniObject3D, we carefully set up four evaluation tracks: a) robust 3D perception, b) novel-view synthesis, c) neural surface reconstruction, and d) 3D object generation. Extensive studies are performed on these four benchmarks, revealing new observations, challenges, and opportunities for future research in realistic 3D vision.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8848d097f07bae98ff01094cf87e59eee94e4692

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets (2024, neutral): “The downside there is the need to fine-tune Stable Diffusion using additional images either by conducting multi-view rendering[Deitkeetal.2023]orusingauxiliarymulti-viewdatasets[Reizen-steinetal.2021;Wuetal.2023;Yuetal.2023a].”
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, methodology): “Mesh Results on OmniObject3D [54].”
Objaverse-XL: A Universe of 10M+ 3D Objects (2023, background): “More recently, IKEA [20] 219 GSO [21] 1K EGAD [22] 2K OmniObject3D [23] 6K PhotoShape [24] 5K ABO [25] 8K Thingi10K [26] 10K 3d-Future [27] 10K ShapeNet [13] 51K Objaverse 1.0 [14] 800K Objaverse-XL 10.2M Segment Anything (SAM) [28] introduced a dataset of one billion object masks used to train a…”
InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models (2024, methodology): “We evaluate the quantitative performance us-ing two public datasets, i.e ., Google Scanned Objects (GSO) [10] and OmniObject3D (Omni3D) [54].”
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (2023, background): “…diffusion models usually work on a single object category and suffer from limited diversity (Jun & Nichol, 2023; Nichol et al., 2022; Gupta et al., 2023; Lorraine et al., 2023; Zhang et al., 2023; Zheng et al., 2023; Ntavelis et al., 2023; Chen et al., 2023b; Cheng et al., 2023; Gao et al., 2022).”
Depth Anything 3: Recovering the Visual Space from Any Views (2025, neutral): “…[98], IRS [93], vKITTI2 [9], BlendedMVS [115], SPRING [56], MVSSynth [38], UnrealStereo4K [123], GTA-SfM [92], TauAgent [27], KenBurns [60], MatrixCity [50], EDEN [47], ReplicaGSO [82], UrbanSyn [32], PointOdyssey [127], Structured3D [125], Objaverse [17], Trel-lis [104], and OmniObject [102].”
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image (2024, methodology): “We assess our model’s efficacy across six zero-shot relative depth benchmarks, including NYUv2 [60], KITTI [13], ETH3D [55], Scan-Net [7], DIODE [64], and OmniObject3D [66].”