SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?

By John McCormac, Ankur Handa, Stefan Leutenegger, Andrew J. Davison. 2017

320 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, methodology): “…61, 62, 79] S+I ScanNet [19] S+I Gibson [62, 79] Replica [62, 72] S+I Synthetic (artist-created) AI2-THOR [39] S+I ARAP [14] I D+R SceneNet-RGBD [52] S+I PBRS [70, 83] S+I CGIntrinsics [45, 70] D InteriorNet [43] S+I D Jiang et al. [36] S+I RobotriX [24] S+I CG-PBR [63, 70] S D+R DeepFurniture…”
Learning to Reconstruct Shapes from Unseen Classes (2018, background): “As handy depth sensors get mature [Izadi et al., 2011], and larger-scale RGB-D datasets become available [McCormac et al., 2017, Silberman et al., 2012, Song et al., 2017], many papers start to estimate depth [Chen et al., 2016, Eigen and Fergus, 2015], surface normals [Bansal and Russell, 2016,…”
Learning Shape Priors for Single-View 3D Completion and Reconstruction (2018, background): “With the development of depth sensors [23] and larger-scale RGB-D datasets [33,42,43], there have also been papers on estimating depth [6,10], surface normals [1,54], and other intrinsic images [25,41] with deep networks.”
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling (2018, background): “Notable attempts include LabelMe3D [46], NYU-D [50], SUN RGB-D [53], KITTI [16], and modern large-scale RGB-D scene datasets [10, 40, 54].”
MarrNet: 3D Shape Reconstruction via 2.5D Sketches (2017, background): “With the development of depth sensors [Izadi et al., 2011] and larger scale RGB-D datasets [Silberman et al., 2012, Song et al., 2017, McCormac et al., 2017], there have also been papers on estimating depth [Chen et al., 2016, Eigen and Fergus, 2015], surface normals [Bansal and Russell, 2016, Wang…”
Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, background): “Alternative options, yet smaller in size, include Al2-THOR (Zhu et al., 2017), SceneNet RGB-D (McCormac et al., 2017), Stanford 3D (Armeni et al., 2016) and Matterport 3D (Chang et al., 2017).”
Kornia: an Open Source Differentiable Computer Vision Library for PyTorch (2019, methodology): “The data used for these experiments was extracted from SceneNet RGB-D dataset [37], containing photorealistic indoor image trajectories.”
Semi-Supervised Semantic Segmentation With Cross-Consistency Training (2020, unknown): “SceneNet [36] Full (5.”
The RobotriX: An Extremely Photorealistic and Very-Large-Scale Indoor Dataset of Sequences with Robot Trajectories and Interactions (2018, background): “In that sense, our target is providing a high resolution dataset with fairly high frame rate (+60 FPS) and three data modalities for each frame: RGB-D, stereo, and 3D (colored point clouds) simulating the particularities of widespread sensors.”
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis (2020, background): “I further indicates that additional pretraining on synthetic data, such as SceneNet [43], should be preferred to deeper backbones, especially if the target dataset is small.”
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization (2018, background): “Conflicting somewhat with the findings of [20], we discovered that pretraining helps significantly even up to 1M images.”
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey (2019, background): “• SceneNet RGB-D: The SceneNet RGB-D dataset is a large indoor synthetic video dataset which consists of 5 million rendered RGB-D images from over 15K trajectories in synthetic layouts with random but physically simulated object poses [115].”
In-Place Scene Labelling and Understanding with Implicit Scene Representation (2021, methodology): “We follow the procedure from SceneNet RGB-D [14], and lock the roll angle with the camera up-vector pointing along the y-axis.”
TartanAir: A Dataset to Push the Limits of Visual SLAM (2020, unknown): “Dataset Sensors/Ground Truth Label Conditions Motion Pattern Seq Num Stereo Flow Depth Lidar Pose Light Weather Season DynObj KITTI [12] � � � � � � Car 22 Virtual KITTI [21] � � � � � � Car 50 EuRoC MAV [14] � � � MAV 11 TUM RGB-D [13] � � � Hand 15 ICL-NUM [28] � � Hand 8 SceneNet [20] � � � � Ran”
Dropout Sampling for Robust Object Detection in Open-Set Conditions (2017, methodology): “nﬂuence of the hyper parameters for the number of forward passes and the required minimum detections per observation. TABLE II PERFORMANCE COMPARISON ON SCENENET RGB-D AT VANILLA SSD REFERENCE SCORES [26] Forward F 1 Score at abs OSE at Passes reference OSE reference F 1 Score vanilla SSD (reference”