Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts

By Ji Hou, Benjamin Graham, Matthias Nießner, Saining Xie. Computer Vision and Pattern Recognition, 2020

342 citations recorded by the source corpus.

The rapid progress in 3D scene understanding has come with growing demand for data; however, collecting and an-notating 3D scenes (e.g. point clouds) are notoriously hard. For example, the number of scenes (e.g. indoor rooms) that can be accessed and scanned might be limited; even given sufficient data, acquiring 3D labels (e.g. instance masks) requires intensive human labor. In this paper, we explore data-efficient learning for 3D point cloud. As a first step towards this direction, we propose Contrastive Scene Contexts, a 3D pre-training method that makes use of both point-level correspondences and spatial contexts in a scene. Our method achieves state-of-the-art results on a suite of benchmarks where training data or labels are scarce. Our study reveals that exhaustive labelling of 3D point clouds might be unnecessary; and remarkably, on ScanNet, even using 0.1% of point labels, we still achieve 89% (instance segmentation) and 96% (semantic segmentation) of the baseline performance that uses full annotations.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/053b96e162724fa10d8639196a99856d60de5214

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Or Litany
2. Jiwen Lu
3. Jie Zhou
4. Yongming Rao
5. Hengshuang Zhao
6. Yifan Wang
7. Yu Zheng
8. Xiuwei Xu

In Computational Mechanics

1. Or Litany
2. Jiwen Lu
3. Jie Zhou
4. Yongming Rao
5. Xiaoyang Wu
6. Xihui Liu
7. Yifan Wang
8. Dávid Rozenberszki

In Geology

1. Or Litany
2. Hengshuang Zhao
3. Xiaoyang Wu
4. Xihui Liu
5. Jiwen Lu
6. Jie Zhou
7. Yongming Rao
8. Dávid Rozenberszki

Citations in the corpus

• Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement (2022, methodology): “To reduce the labor of human annotation, weakly-supervised methods [21, 22, 29, 33], semi-supervised methods [41, 49] and unsupervised pretraining methods [12,30,45,47] have been proposed recently.”
• Unsupervised Point Cloud Pre-training via Occlusion Completion (2020, neutral): “These have been recently outperformed by self-supervised objectives [42, 56, 2, 44, 20, 61].”
• Language-Grounded Indoor 3D Semantic Segmentation in the Wild (2022, methodology): “Figure 8 shows qualitative visualizations for the downstream task of 3D instance segmentation in comparison to training from scratch and CSC [20].”
• Point Transformer V3: Simpler, Faster, Stronger (2023, methodology): “9, we evaluate the performance of PTv3 on the ScanNet data efficient [30] benchmark.”