OccuSeg: Occupancy-Aware 3D Instance Segmentation

By Lei Han, Tian Zheng, Lan Xu, Lu Fang. Computer Vision and Pattern Recognition, 2020

301 citations recorded by the source corpus.

3D instance segmentation, with a variety of applications in robotics and augmented reality, is in large demands these days. Unlike 2D images that are projective observations of the environment, 3D models provide metric reconstruction of the scenes without occlusion or scale ambiguity. In this paper, we define “3D occupancy size”, as the number of voxels occupied by each instance. It owns advantages of robustness in prediction, on which basis, OccuSeg, an occupancy-aware 3D instance segmentation scheme is proposed. Our multi-task learning produces both occupancy signal and embedding representations, where the training of spatial and feature embeddings varies with their difference in scale-aware. Our clustering scheme benefits from the reliable comparison between the predicted occupancy size and the clustered occupancy size, which encourages hard samples being correctly clustered and avoids over segmentation. The proposed approach achieves state-of-theart performance on 3 real-world datasets, i.e. ScanNetV2, S3DIS and SceneNN, while maintaining high efficiency.

Subfields: Aerospace Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d846b06c98070ed1f9ffc4b5a10bd533640848ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Unseen Object Instance Segmentation for Robotic Environments (2020, background): “[54] proposed an occupancy term which greatly aids supervoxel clustering, leading to strong results.”
Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, methodology): “, centers [26], bounding boxes [7], occupancy [18]), and heuristics for clustering the”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, background): “OccuSeg [Han et al., 2020] proposed an occupancy-aware 3D instance segmen-”
Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, background): “In the past year, sparse convolutional networks [20, 12] stand out as a promising approach to standardize deep learning for point clouds, due to its computational efﬁciency and state-of-the-art performance for 3D scene understanding tasks [12, 25, 34].”
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion (2020, unknown): “OccuSeg (Han et al. 2020) proposes a 3D voxel projection-based segmentation network with voxel occupancy size regression and owns advantages of robustness in prediction.”
A Survey on Multimodal Large Language Models for Autonomous Driving (2023, background): “Beyond vision and language, researchers also proposed models for other modalities, such as audio, speech, and 3D data.”
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020, background): “There are also 3D models tailored for specific tasks such as detection [38, 39, 41, 47–49, 70, 73] and instance segmentation [16, 21, 23, 71] built upon these modules.”
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation (2020, methodology): “OccuSeg [13], SSCN [12] and SEGCloud [30] follow this line to utilize the voxel partition and apply regular 3D convolutions for LiDAR segmentation.”
SoftGroup for 3D Instance Segmentation on Point Clouds (2022, unknown): “Han et al. [9] introduce OccuSeg that performs graph-based clustering guided by object occupancy signal for more accurate segmentation outputs.”
Language-Grounded Indoor 3D Semantic Segmentation in the Wild (2022, background): “The introduction of several large-scale real-world 3D datasets [10,4,1] has led to rapid developments in data-driven 3D deep learning techniques, with an emphasis on point-and sparse-voxel-based approaches [15,48,9,52,18].”
OpenScene: 3D Scene Understanding with Open Vocabularies (2022, background): “Most prior work focuses on training models with ground-truth 3D labels [11, 20, 22, 23, 25, 31, 41, 42, 45, 48, 54].”
OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, background): “3D instance segmentation goes a step further by distinguishing multiple objects belonging to the same semantic category, predicting individual masks for each object instance [11, 14, 22, 24, 31, 37, 55, 60, 62, 67].”