SBNet: Sparse Blocks Network for Fast Inference

By Mengye Ren, Andrei Pokrovsky, Binh Yang, Raquel Urtasun. 2018

192 citations recorded by the source corpus.

Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications. For many problems such as object detection and semantic segmentation, we are able to obtain a low-cost computation mask, either from a priori problem knowledge, or from a low-resolution segmentation network. We show that such computation masks can be used to reduce computation in the high-resolution main network. Variants of sparse activation CNNs have previously been explored on small-scale tasks and showed no degradation in terms of object classification accuracy, but often measured gains in terms of theoretical FLOPs without realizing a practical speedup when compared to highly optimized dense convolution implementations. In this work, we leverage the sparsity structure of computation masks and propose a novel tiling-based sparse convolution algorithm. We verified the effectiveness of our sparse CNN on LiDAR-based 3D object detection, and we report significant wall-clock speed-ups compared to dense convolution without noticeable loss of accuracy.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, methodology): “Existing sampling strategies include 1) making use of the intrinsic sparsity of the input [143]; 2) predicting the positions of zero elements on the output [144], [145]; and 3) estimating the saliency of pixels [146], [147], [148].”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Several other works have reused camera-based algorithms and trained efficient networks for 3D object detection on 2D BV images [170]–[173].”
S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search (2019, background): “They attempted to add extra controllers to select which computations are executed [5, 3, 34, 25, 15, 7, 46, 41, 42].”
Learning Instance-wise Sparsity for Accelerating Deep Models (2019, methodology): “[Almahairi et al. , 2016; Dong et al. , 2017; Ren et al. , 2018] applied the dynamic strategy on the activations of feature maps in neural networks.”
Dynamic Channel Pruning: Feature Boosting and Suppression (2018, background): “Ren et al. (2018) assumed the availability of a priori spatial sparsity in the input image, and accelerated the convolutional layer by computing non-sparse regions.”
Batch-shaping for learning conditional channel gated networks (2019, background): “Similarly, tiling-based sparse convolution algorithm was proposed (Ren et al., 2018) which yields a significant speed-up in terms of wall-clock time.”
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection (2021, unknown): “SBNet [42] adopts an offline road map or a mask to filter out ignore region.”
Pay Attention to Features, Transfer Learn Faster CNNs (2020, methodology): “There are methods that dynamically select which paths to evaluate in a network dependent on the input (Figurnov et al., 2017; Dong et al., 2017b; Bolukbasi et al., 2017; Lin et al., 2017; Shazeer et al., 2017; Wu et al., 2018; Ren et al., 2018).”
BlockCopy: High-Resolution Video Processing with Block-Sparse Feature Propagation and Online Policies (2021, methodology): “To this end, SBNet [35] proposes to use partly overlapping blocks and applies this for 3D object detection.”
Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference (2019, background): “One of the only works showing practical speedup with spatially conditional execution is SBNet [36].”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, methodology): “SBNet [21] utilizes the sparsity in road mask to speed up 3D detection by > 2 times.”
Focal Sparse Convolutional Networks for 3D Object Detection (2022, methodology): “, kernel shape adaption [10, 41, 64], and input attention mask [34, 42, 45].”
Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation (2020, methodology): “SBNet [32] is a general method to accelerate sparse convolution in GPU.”