Objaverse: A Universe of Annotated 3D Objects

By Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, Ali Farhadi. Computer Vision and Pattern Recognition, 2022

1,762 citations recorded by the source corpus.

Massive data corpora like WebText, Wikipedia, Conceptual Captions, WebImageText, and LAION have propelled recent dramatic progress in AI. Large neural models trained on such datasets produce impressive results and top many of today's benchmarks. A notable omisslion within this family of large-scale datasets is 3D data. Despite considerable interest and potential applications in 3D vision, datasets of high-fidelity 3D models continue to be mid-sized with limited diversity of object categories. Addressing this gap, we present Objaverse 1.0, a large dataset of objects with 800K + (and growing) 3D models with descriptive captions, tags, and animations. Objaverse improves upon present day 3D repositories in terms of scale, number of categories, and in the visual diversity of instances within a category. We demonstrate the large potential of Objaverse via four diverse applications: training generative 3D models, improving tail category segmentation on the LVIS benchmark, training open-vocabulary object-navigation models for Embodied AI, and creating a new benchmark for robustness analysis of vision models. Objaverse can open new directions for research and enable new applications across the field of AI.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1b31dbf44e68b698120552366df03e6e35a1e428

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Zero-1-to-3: Zero-shot One Image to 3D Object (2023, methodology): “Such a trend can also be found on RTMV in Figure 6, even though the scenes are out-of-distribution compared to the Objaverse dataset.”
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion (2024, methodology): “Similar to SVD-MV [2], we render views of a subset of the Objaverse dataset [9] consisting of 150K curated CC-licensed 3D objects from the original dataset.”
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets (2023, methodology): “We finetuned our SVD model on two datasets, where the SVD model takes a single image and outputs a sequence of multi-view images: (i) A subset of Obja-verse [15] consisting of 150K curated and CC-licensed synthetic 3D objects from the original dataset [15].”
Structured 3D Latents for Scalable and Versatile 3D Generation (2024, neutral): “Therefore, we include only the objects from Sketchfab (also known as Objaver-seV1 [7]) and GitHub in our training dataset and perform a thorough filtering process to clean the dataset.”
MVDream: Multi-view Diffusion for 3D Generation (2023, methodology): “To ensure the multi-view consistency of our model, we render a set of multi-view images from a real 3D dataset, namely objaverse (Deitke et al., 2023).”
PointLLM: Empowering Large Language Models to Understand Point Clouds (2023, methodology): “We use the human-annotated split of the Cap3D [29] dataset as our source of data, which contains point clouds of various objects from the Objaverse [9] dataset with human-annotated captions.”
LRM: Large Reconstruction Model for Single Image to 3D (2023, positive): “…more than 500 million learnable parameters, and it is trained on approximately one million 3D shapes and video data across diverse categories (Deitke et al., 2023; Yu et al., 2023); this is substantially larger than recent methods that apply relatively shallower networks and smaller datasets…”
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action (2023, methodology): “For self-supervised 3D and embodiment pre-training, we use CroCo [194] for cross-view generation and denoising; Objaverse [40] for view synthesis; and random trajectories in ProcTHOR [39] and Habitat [157] for the next action and frame predictions.”
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization (2023, methodology): “To train our reconstruction module, we utilize the LVIS subset of the Objaverse [11] dataset, which consists of 46k 3D models across 1,156 categories.”
CameraCtrl: Enabling Camera Control for Text-to-Video Generation (2024, neutral): “Indeed, datasets with computer-generated imagery, such as Objaverse (Deitke et al., 2023), exhibit diverse camera distributions since we can control the camera parameters during the rendering process.”
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image (2024, background): “( See Supp. for visualization) 3)Background-free Object: Objaverse [8,45] is a massive dataset of over 10 million 3D objects.”
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (2023, methodology): “Zero-1-to-3 XL (Liu et al., 2023b; Deitke et al., 2023b) is adopted as the 2D diffusion prior.”
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting (2024, methodology): “We train two versions of our GS-LRM on two large-scale datasets including Objaverse [19] and Real-Estate10K [76], separately for object and scene reconstruction tasks, using the same transformer architecture with minimal domain-specific parameter changes.”
Wonder3D: Single Image to 3D Using Cross-Domain Diffusion (2023, methodology): “We train our model on the LVIS subset of the Objaverse dataset [10], which comprises approximately 30k+ objects following a cleanup process.”
InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models (2024, methodology): “The availability of large-scale 3D datasets [8, 9] enables training highly generalizable reconstruction models for feed-forward image-to-3D creation.”
An Embodied Generalist Agent in 3D World (2023, neutral): “To facilitate object-level grounding of detailed object attributes, we leverage Cap3D (Luo et al., 2023), which contains language descriptions for objects in Objaverse (Deitke et al., 2023).”
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model (2023, methodology): “In the context of object and camera poses, it’s worth noting that the Objaverse dataset [4] does not consistently align objects in a canonical pose, despite them typically being oriented along the gravity axis.”
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset (2024, neutral): “Examples include ImageNet [11], Kitti [18], Ego4D [19] and LAION [46] in computer vision, Common Crawl [43] and The Pile [17] in natural language processing, and ShapeNet [5] and Objaverse [9, 10] in 3D modeling.”
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation (2025, neutral): “Due to the viewpoint bias and multi-view inconsistency inherent in training-free image diffusion models, multi-view image diffusion was developed to alleviate these issues by utilizing large-scale 3D datasets, such as objaverse and objaverse-xl [20, 19].”
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model (2023, methodology): “Both of these stages are fine-tuned/trained with multi-view rendered images of 3D objects (Deitke et al., 2023b).”