Recurrent Slice Networks for 3D Segmentation of Point Clouds

By Qiangui Huang, Weiyue Wang, Ulrich Neumann. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

484 citations recorded by the source corpus.

Point clouds are an efficient data format for 3D data. However, existing 3D segmentation methods for point clouds either do not model local dependencies [21] or require added computations [14, 23]. This work presents a novel 3D segmentation framework, RSNet1, to efficiently model local structures in point clouds. The key component of the RSNet is a lightweight local dependency module. It is a combination of a novel slice pooling layer, Recurrent Neural Network (RNN) layers, and a slice unpooling layer. The slice pooling layer is designed to project features of unordered points onto an ordered sequence of feature vectors so that traditional end-to-end learning algorithms (RNNs) can be applied. The performance of RSNet is validated by comprehensive experiments on the S3DIS[1], ScanNet[3], and ShapeNet [34] datasets. In its simplest form, RSNets surpass all previous state-of-the-art methods on these benchmarks. And comparisons against previous state-of-the-art methods [21, 23] demonstrate the efficiency of RSNets.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Geology

View Paper: https://www.semanticscholar.org/paper/2a33d0939cb4f9200d401a453146bb120ee3b403

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Geology

Citations in the corpus

DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing (2019, unknown): “3 RS-Net [15] 81.”
Point-Voxel CNN for Efficient 3D Deep Learning (2019, methodology): “We adopt PointNet [30], RSNet [13], PointNet++ [32] (with multi-scale grouping), DGCNN [43], SpiderCNN [46] and PointCNN [23] as our point-based baselines.”
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (2019, background): “These modules can be generally categorized as: 1) neighbouring feature pooling [44, 32, 21, 70, 69], 2) graph message passing [57, 48, 55, 56, 5, 22, 34], 3) kernel-based convolution [49, 20, 60, 29, 23, 24, 54, 38], and 4) attentionbased aggregation [61, 68, 66, 42].”
Deep Learning for 3D Point Clouds: A Survey (2019, unknown): “[212] proposed a lightweight local dependency model-”
Depth-aware CNN for RGB-D Segmentation (2018, background): “y, many works [11,23] attempt to solve the problem with 3D CNNs. However, the volumetric representation prevents scaling up due to high memory and computation cost. Recently, deep learning frameworks [4,24,25,26,27] on point cloud are introduced to address the limitations of 3D volume. Qi et al. [4]”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, unknown): “RSNet [15] 81.”
Modeling Point Clouds With Self-Attention and Gumbel Subset Sampling (2019, methodology): “RSNet [13] 56.47 - - SPGraph [20] 62 To further analyze the performance between PointCNN and our method, we compare per-class IoU and mean per-class accuracy (mAcc) on AREAS and Area 5.”
PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing (2019, unknown): “2 RSNet [10] - 66.”
Relation-Shape Convolutional Neural Network for Point Cloud Analysis (2019, unknown): “6 RS-Net [12] 81.”
Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution (2018, background): “RSNet (Huang, Wang, and Neumann 2018) is combines a novel slice pooling layer, Recurrent Neural Network (RNN) layers, and a slice unpooling layer, to better propogate features among different parts.”
Monte Carlo convolution for learning on non-uniformly sampled point clouds (2018, background): “Follow up work has proposed several extensions [Atzmon et al. 2018; Graham and van der Maaten 2017; Groh et al. 2018; Huang et al. 2018; Li et al. 2018; Shen et al. 2017; Su et al. 2018; Wang et al. 2017, 2018; Xu et al. 2018; Yu et al. 2018].”
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning (2019, methodology): “Some methods are specially design to be lightweight for real-time applications [3], or to combine with recurrent neural network [13] and sequence model [21].”
Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, background): “To address it, a number of approaches [38; 57; 43; 31; 55; 49; 26; 17] have been proposed recently.”
MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation (2019, methodology): “Later methods [16, 8, 10] model local dependencies with various tricks, but the number of points processed by the network is still quite limited (e.g., 4 , 096 ).”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, methodology): “Instead of using point-based methods like [27], [28], [57], [58], [59], [60], [61] for extracting point-wise features from the point cloud, as show in the left part of Fig.”
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud (2018, background): “The follow-up works [28, 12] further improve the extracted feature quality by considering the local structures in point clouds.”
Associatively Segmenting Instances and Semantics in Point Clouds (2019, unknown): “The following works Point-Net++ [28], RSNet [13], DGCNN [36] and PointCNN [17] further focus on exploring the local context and hierarchical learning architectures.”
Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving (2019, methodology): “With the development of deep learning on raw point clouds [28, 29, 13], several detection approaches only based on raw LiDAR data are also proposed.”