3DmFV: Three-Dimensional Point Cloud Classification in Real-Time Using Convolutional Neural Networks

By Yizhak Ben-Shabat, M. Lindenbaum, Anath Fischer. 2018

224 citations recorded by the source corpus.

Modern robotic systems are often equipped with a direct three-dimensional (3-D) data acquisition device, e.g., LiDAR, which provides a rich 3-D point cloud representation of the surroundings. This representation is commonly used for obstacle avoidance and mapping. Here, we propose a new approach for using point clouds for another critical robotic capability, semantic understanding of the environment (i.e., object classification). Convolutional neural networks (CNNs), that perform extremely well for object classification in 2-D images, are not easily extendible to 3-D point clouds analysis. It is not straightforward due to point clouds’ irregular format and a varying number of points. The common solution of transforming the point cloud data into a 3-D voxel grid needs to address severe accuracy versus memory size tradeoffs. In this letter, we propose a novel, intuitively interpretable, 3-D point cloud representation called 3-D modified Fisher vectors. Our representation is hybrid as it combines a coarse discrete grid structure with continuous generalized Fisher vectors. Using the grid enables us to design a new CNN architecture for real-time point cloud classification. In a series of performance analysis experiments, we demonstrate competitive results or even better than state of the art on challenging benchmark datasets while maintaining robustness to various data corruptions.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Geology

Citations in the corpus

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, background): “DeepSet[30] 3DmFV[3] Set Transformer[12] PointNet[15] SpiderCNN[29] Tensor Field +z[20] PointNet++[16] SE(3)-Transf.”
Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, methodology): “ModelNet40 OA mAcc OA mAcc 3DmFV [2] 63.0 58.”
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning (2019, methodology): “A wide body of works [24, 26, 12, 17, 19, 35, 39, 3] have recently been proposed to solve this problem, demonstrating state-of-the-art performance in scene understanding with point cloud data.”
Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline (2021, unknown): “…2017; Zaheer et al., 2017; Klokov & Lempitsky, 2017; Xu et al., 2018; Atzmon et al., 2018; Wang et al., 2018a; Li et al., 2018a; Groh et al., 2018; Ben-Shabat et al., 2018; Xie et al., 2018; Li et al., 2018b; Liu et al., 2019a; Thomas et al., 2019; Komarichev et al., 2019; Liu et al., 2019b; Yan…”
ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics (2019, methodology): “[4] use a grid of spherical Gaussians with Fisher vectors to describe points.”
PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds (2021, methodology): “Another technique is to quantize the 3D space and map points into regular voxels [39, 33, 3, 34], where 3D convolutions can be applied.”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, methodology): “In the case of voxel-based methods, the points are projected on 3D grids in Euclidean space [23, 29, 3].”
SGAS: Sequential Greedy Architecture Search (2019, unknown): “3DmFV-Net [4] 91.”