OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets

By Zhengqin Li, Ting Yu, Shen Sang, Sarah Wang, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Milovs Havsan, Ravi Ramamoorthi and 1 more. 2020

113 citations recorded by the source corpus.

We propose a novel framework for creating large-scale photorealistic datasets of indoor scenes, with ground truth geometry, material, lighting and semantics. Our goal is to make the dataset creation process widely accessible, transforming scans into photorealistic datasets with high-quality ground truth for appearance, layout, semantic labels, high quality spatially-varying BRDF and complex lighting, including direct, indirect and visibility components. This enables important applications in inverse rendering, scene understanding and robotics. We show that deep networks trained on the proposed dataset achieve competitive performance for shape, material and lighting estimation on real images, enabling photorealistic augmented reality applications, such as object insertion and material editing. We also show our semantic labels may be used for segmentation and multi-task learning. Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes. The dataset and all the tools to create such datasets will be made publicly available.1

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology, Evaluation & Benchmarks, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

In Evaluation & Benchmarks

In 3D Vision & Pose

Citations in the corpus

Habitat 2.0: Training Home Assistants to Rearrange their Habitat (2021, methodology): “ReplicaCAD like other efforts such as OpenRooms [61] and Scan2cad [62] creates a version of the scene with decoupled 3D models suited for physical simulation and interaction.”
ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, methodology): “OpenRooms [33] is similar to HM3D in terms of the source of the data (3D scans) and dataset size.”