The RobotriX: An Extremely Photorealistic and Very-Large-Scale Indoor Dataset of Sequences with Robot Trajectories and Interactions

By Alberto Garcia-Garcia, Pablo Martinez-Gonzalez, Sergiu Oprea, John Alejandro Castro-Vargas, Sergio Orts, Jose García Rodríguez, Alvaro Jover-Alvarez. 2018

37 citations recorded by the source corpus.

Enter the RobotriX, an extremely photorealistic indoor dataset designed to enable the application of deep learning techniques to a wide variety of robotic vision problems. The RobotriX consists of hyperrealistic indoor scenes which are explored by robot agents which also interact with objects in a visually realistic manner in that simulated world. Photorealistic scenes and robots are rendered by Unreal Engine into a virtual reality headset which captures gaze so that a human operator can move the robot and use controllers for the robotic hands; scene information is dumped on a per-frame basis so that it can be reproduced offline using UnrealCV to generate raw data and ground truth labels. By taking this approach, we were able to generate a dataset of 38 semantic classes across 512 sequences totaling 8M stills recorded at +60 frames per second with full HD resolution. For each frame, RGB-D and 3D information is provided with full annotations in both spaces. Thanks to the high quality and quantity of both raw information and annotations, the RobotriX will serve as a new milestone for investigating 2D and 3D robotic vision tasks with large-scale data-driven techniques.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Robotics & Control

Citations in the corpus

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, background): “…the color image. alistic synthetic datasets and interactive simulation environments that have spurred rapid progress towards the goal of holistic indoor scene understanding [6, 7, 9, 10, 14, 15, 19, 22, 24, 31, 33, 36, 37, 39, 43, 44, 45, 46, 49, 52, 61, 62, 63, 65, 70, 72, 75, 76, 79, 83, 84].”
The Replica Dataset: A Digital Replica of Indoor Spaces (2019, background): “On the smaller scale with only 16 scenes but with more realistic appearance is the RobotriX dataset [13].”
3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics (2020, background): “Over the past years, a large number of RGB-D benchmarks have been constructed and made publicly available [21, 3, 27, 36, 43, 33, 19, 6, 9, 18, 26, 23, 49, 24, 4, 32, 35, 42, 11, 17, 47, 38, 2].”
iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks (2021, unknown): “More physically realistic are the VR interactions with UnrealROX/RobotriX [32, 33] and ThreeDWorld [7], based on Unreal [34] and Nvidia Physx [20].”