SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud

By Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, Kurt Keutzer. 2018

726 citations recorded by the source corpus.

Earlier work demonstrates the promise of deep-learning-based approaches for point cloud segmentation; however, these approaches need to be improved to be practically useful. To this end, we introduce a new model SqueezeSegV2. With an improved model structure, SqueezeSetV2 is more robust against dropout noises in LiDAR point cloud and therefore achieves significant accuracy improvement. Training models for point cloud segmentation requires large amounts of labeled data, which is expensive to obtain. To sidestep the cost of data collection and annotation, simulators such as GTA-V can be used to create unlimited amounts of labeled, synthetic data. However, due to domain shift, models trained on synthetic data often do not generalize well to the real world. Existing domain-adaptation methods mainly focus on images and most of them cannot be directly applied to point clouds. We address this problem with a domain-adaptation training pipeline consisting of three major components: 1) learned intensity rendering, 2) geodesic correlation alignment, and 3) progressive domain calibration. When trained on real data, our new model exhibits segmentation accuracy improvements of 6.0-8.6% over the original SqueezeSeg. When training our new model on synthetic data using the proposed domain adaptation pipeline, we nearly double test accuracy on real-world data, from 29.0% to 57.4%. Our source code and synthetic dataset are open sourced11https://github.com/xuanyuzhou98/SqueezeSegV2

Subfields: Artificial Intelligence, Civil and Structural Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Civil and Structural Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Visual Transformers: Token-based Image Representation and Processing for Computer Vision (2020, methodology): “Attention was first used to modulate the feature map: some attention values are computed from the input and multiplied to the feature map as in [9, 7, 8, 10].”
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search (2018, methodology): “It is originally designed for classification, but later extended to object detection [22] and LiDAR point-cloud segmentation [24, 26].”
Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs (2018, methodology): “While SqueezeNet is designed for image classification, later models, including SqueezeDet [15] and SqueezeSeg [16, 17], extend the scope to object detection and point-cloud segmentation.”
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences (2019, methodology): “SqueezeSeg [60, 61] also discretizes the point cloud in a way that makes it possible to apply 2D convolutions to the point cloud data exploiting the sensor geometry of a rotating LiDAR.”
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation (2020, methodology): “SqueezeSeg [42], Darknet [2], Squeeze-Segv2 [43], and RangeNet++ [27] utilize the spherical projection mechanism, which converts the point cloud to a frontal-view image or a range image, and adopt the 2D convolution network on the pseudo image for point cloud seg-2 mentation.”
RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation (2019, methodology): “[21], [22] used the provided bounding box of the KITTI dataset [7].”
SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds (2020, neutral): “12.9 13.1 0.9 85.4 26.9 54.3 4.5 57.4 29.0 60.0 24.3 53.7 17.5 24.5 29.5 SqueezeSeg-CRF [6] 68.3 18.1 5.1 4.1 4.8 16.5 17.3 1.2 84.9 28.4 54.7 4.6 61.5 29.2 59.6 25.5 54.7 11.2 36.3 30.8 SqueezeSegV2 [10] 81.8 18.5 17.9 13.4 14.0 20.1 25.1 3.9 88.6 45.8 67.6 17.7 73.7 41.1 71.8 35.8 60.2 20.2 36.3 3”
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion (2020, background): “Previous works projected all points scanned by the rotating LiDAR onto 2D images by plane projection (Lawin et al. 2017; Boulch, Le Saux, and Audebert 2017; Tatarchenko et al. 2018) or spherical projection (Wu et al. 2018, 2019).”
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation (2020, background): “SqueezeSeg [35] and SqueezeSegV2 [36] used spherical projections to represent point clouds for a light 2D semantic segmentation network, which is able to achieve real-time performance.”
Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “spherical [183], [184], [185], volumetric [166], [186], [187], permutohedral lattice [188], [189], and hybrid representations [190], [191], as shown in Fig.”
SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation (2020, methodology): “Previous methods based on spherical projection [56,58,30] treat projected LiDAR images as RGB images and process them with standard convolution as”
2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (2022, background): “Previous works project all points scanned by the rotating LiDAR onto 2D images by plane projection [34–36], spherical projection [37,38] or both [39].”
SqueezeWave: Extremely Lightweight Vocoders for On-device Speech Synthesis (2020, background): “Previous work (Iandola et al., 2016; Wu et al., 2016; 2017; 2018a;b; Yang et al., 2018; Gholami et al., 2018; Wu, 2019; Wu et al., 2019; Dai et al., 2019) have shown that optimizing the neural network architecture can lead to significant efficiency improvement in many applications.”
(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network (2021, methodology): “Conv (12x12x12) [4, 32] Conv (2x2x2) [32, 32]”
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation (2020, methodology): “homogeneity homogeneous DA dS = dT source and target data are observed in the same space [11, 23, 36, 44–106, 113–151] heterogeneous DA dS 6= dT source and target data are observed in different spaces [107–112]”
Synthetic Data for Deep Learning (2019, methodology): “[643,644] added a LiDAR simulator to Grand Theft Auto V and collected a synthetic dataset from the game.”
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (2019, methodology): “1) pointbased methods [43, 26, 49, 44, 51] and 2) projection based approaches [58, 59, 3, 40], and Figure 6 shows some qualitative results of RandLA-Net on the validation split.”