RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation

By Jianyun Xu, Ruixiang Zhang, Jian Dou, Yushi Zhu, Jie Sun, Shiliang Pu. IEEE International Conference on Computer Vision, 2021

364 citations recorded by the source corpus.

Point clouds can be represented in many forms (views), typically, point-based sets, voxel-based cells or range-based images(i.e., panoramic view). The point-based view is geometrically accurate, but it is disordered, which makes it difficult to find local neighbors efficiently. The voxel-based view is regular, but sparse, and computation grows cubicly when voxel resolution increases. The range-based view is regular and generally dense, however spherical projection makes physical dimensions distorted. Both voxel-and range-based views suffer from quantization loss, especially for voxels when facing large-scale scenes. In order to utilize different view’s advantages and alleviate their own shortcomings in fine-grained segmentation task, we propose a novel range-point-voxel fusion network, namely RPVNet. In this network, we devise a deep fusion framework with multiple and mutual information interactions among these three views, and propose a gated fusion module (termed as GFM), which can adaptively merge the three features based on concurrent inputs. Moreover, the proposed RPV interaction mechanism is highly efficient, and we summarize it to a more general formulation. By leveraging this efficient interaction and relatively lower voxel resolution, our method is also proved to be more efficient. Finally, we evaluated the proposed model on two large-scale datasets, i.e., SemanticKITTI and nuScenes, and it shows state-of-the-art performance on both of them. Note that, our method currently ranks 1st on SemanticKITTI leaderboard without any extra tricks.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/5bf7c535e6cc94c9834a7c31cc3d5ed9376757c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

  1. Or Litany
  2. Chaoda Zheng
  3. Dávid Rozenberszki
  4. Xu Yan
  5. Angela Dai
  6. Matthias Nießner
  7. Leonidas J. Guibas
  8. Charles Ruizhongtai Qi

In Computational Mechanics

  1. Or Litany
  2. Chaoda Zheng
  3. Dávid Rozenberszki
  4. Xu Yan
  5. Zhen Li
  6. Angela Dai
  7. Ruimao Zhang
  8. Jiantao Gao

In Geology

  1. Or Litany
  2. Dávid Rozenberszki
  3. Chaoda Zheng
  4. Xu Yan
  5. Angela Dai
  6. Matthias Nießner
  7. Leonidas J. Guibas
  8. Charles Ruizhongtai Qi

Citations in the corpus