OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction

By Yunpeng Zhang, Zhengbiao Zhu, Dalong Du. IEEE International Conference on Computer Vision, 2023

382 citations recorded by the source corpus.

The vision-based perception for autonomous driving has undergone a transformation from the bird-eye-view (BEV) representations to the 3D semantic occupancy. Compared with the BEV planes, the 3D semantic occupancy further provides structural information along the vertical direction. This paper presents OccFormer, a dual-path transformer network to effectively process the 3D volume for semantic occupancy prediction. OccFormer achieves a long-range, dynamic, and efficient encoding of the camera-generated 3D voxel features. It is obtained by decomposing the heavy 3D processing into the local and global transformer pathways along the horizontal plane. For the occupancy decoder, we adapt the vanilla Mask2Former for 3D semantic occupancy by proposing preserve-pooling and class-guided sampling, which notably mitigate the sparsity and class imbalance. Experimental results demonstrate that OccFormer significantly outperforms existing methods for semantic scene completion on SemanticKITTI dataset and for LiDAR semantic segmentation on nuScenes dataset. Code is available at https://github.com/zhangyp15/OccFormer.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2ac4fb1e431276536d5eb5313ce6001cdbc7b603

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Tao Jiang
2. Yue Wang
3. Hang Zhao
4. Yilun Wang
5. Wotao Yin
6. Yin Zhang

In Computer Graphics and Computer-Aided Design

1. Yue Wang
2. Hang Zhao
3. Yilun Wang

In Computer Vision and Pattern Recognition

1. Yue Wang
2. Yilun Wang
3. Hang Zhao
4. Tao Jiang
5. Xiaoyu Tian
6. Longfei Yun
7. Luc Van Gool
8. Xiaogang Wang

Citations in the corpus

• Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, positive): “Despite recent advancements in 3D occupancy prediction [5, 16, 53], there is a notable absence of high-quality datasets together with benchmarks.”