JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

By Lin Zhao, Wenbing Tao. 2019

129 citations recorded by the source corpus.

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at this https URL.

Subfields: Computational Mechanics, Environmental Engineering, Geology, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yulan Guo
2. Mohammed Bennamoun
3. Hao Liu
4. Ronghua Liang
5. Yanghui Xu
6. Hanyun Wang
7. Jiazhou Chen
8. Liangliang Nan

In Environmental Engineering

1. Mohammed Bennamoun
2. Hao Liu
3. Yulan Guo
4. Hanyun Wang
5. Liangliang Nan
6. Li Liu
7. Qingyong Hu

In Geology

1. Yulan Guo
2. Mohammed Bennamoun
3. Hao Liu
4. Hanyun Wang
5. Liangliang Nan
6. Li Liu
7. Qingyong Hu

In 3D Vision & Pose

1. Yulan Guo
2. Mohammed Bennamoun
3. Hao Liu
4. Ronghua Liang
5. Yanghui Xu
6. Hanyun Wang
7. Jiazhou Chen
8. Liangliang Nan

Citations in the corpus

• 3D Instance Segmentation of MVS Buildings (2021, background): “JSNet [41] and ASIS [10] both learn the instance embedding space and combine semantic features and instance features of the point clouds to jointly improve the”
• Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “[242] proposed JSNet to achieve both semantic and instance segmentation.”