ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data

By Afshin Dehghan, Gilad Baruch, Zhuoyuan Chen, Yuri Feigin, Peter Fu, Thomas Gebauer, Daniel Kurz, Tal Dimry, Brandon Joffe, Arik Schwartz and 1 more. NeurIPS Datasets and Benchmarks, 2021

470 citations recorded by the source corpus.

Scene understanding is an active research area. Commercial depth sensors, such as Kinect, have enabled the release of several RGB-D datasets over the past few years which spawned novel methods in 3D scene understanding. More recently with the launch of the LiDAR sensor in Apple's iPads and iPhones, high quality RGB-D data is accessible to millions of people on a device they commonly use. This opens a whole new era in scene understanding for the Computer Vision community as well as app developers. The fundamental research in scene understanding together with the advances in machine learning can now impact people's everyday experiences. However, transforming these scene understanding methods to real-world experiences requires additional innovation and development. In this paper we introduce ARKitScenes. It is not only the first RGB-D dataset that is captured with a now widely available depth sensor, but to our best knowledge, it also is the largest indoor scene understanding data released. In addition to the raw and processed data from the mobile device, ARKitScenes includes high resolution depth maps captured using a stationary laser scanner, as well as manually labeled 3D oriented bounding boxes for a large taxonomy of furniture. We further analyze the usefulness of the data for two downstream tasks: 3D object detection and color-guided depth upsampling. We demonstrate that our dataset can help push the boundaries of existing state-of-the-art methods and it introduces new challenges that better represent real-world scenarios.

Subfields: Aerospace Engineering, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/19c68c1e0cf0d47b96bd448e0ade1c4add0601d6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Environmental Engineering

In Geology

Citations in the corpus

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision (2023, unknown): “RealEstate10k [42], ARKitScenes [5], ScanNet [11], and the high-resolution ScanNet++ [36] improve this with a broad range of detailed indoor scenes.”
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI (2021, background): “[21] have recently released reconstructions of 1,661 scenes but most are single room-scale regions, from rented homes in three European cities.”
Qwen3-VL Technical Report (2025, unknown): “…understanding: RefCOCO/+/g (Kazemzadeh et al., 2014; Mao et al., 2016), ODinW-13 (Li et al., 2022), CountBench (Paiss et al., 2023), ARKitScenes (Baruch et al., 2021), Hypersim (Roberts et al., 2021), SUN RGB-D (Song et al., 2015), ERQA (Team et al., 2025), VSIBench (Yang et al., 2025b),…”
Depth Anything 3: Recovering the Visual Space from Any Views (2025, unknown): “AriaDigitalTwin [63] 237 Synthetic AriaSyntheticENV [63] 99950 Synthetic ArkitScenes [5] 4388 LiDAR BlendedMVS [115] 503 3D Recon Co3dv2 [68] 30616 Colmap DL3DV [53] 6379 Colmap HyperSim [69] 344 Synthetic MapFree [3] 921 Colmap MegaDepth [51] 268 Colmap MegaSynth [40] 6049 Synthetic MvsSynth [38]…”
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model (2024, methodology): “To address this gap, we develop SpatialRGPT-Bench, a spatial reasoning VQA benchmark using data from both urban (nuScenes [54], KITTI [55]) and indoor (SUNRGBD [56], ARKitScenes [57]) environments, as well as simulated scenes (Hypersim [58]).”
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2024, unknown): “Our benchmark aggregates existing 3D indoor scene understanding and reconstruction datasets: ScanNet [19], ScanNet++ [97], and ARKitScenes [5].”