CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

By Yuhong Li, Xiaofan Zhang, Deming Chen. 2018

1,638 citations recorded by the source corpus.

We propose a network for Congested Scene Recognition called CSRNet to provide a data-driven and deep learning method that can understand highly congested scenes and perform accurate count estimation as well as present high-quality density maps. The proposed CSRNet is composed of two major components: a convolutional neural network (CNN) as the front-end for 2D feature extraction and a dilated CNN for the back-end, which uses dilated kernels to deliver larger reception fields and to replace pooling operations. CSRNet is an easy-trained model because of its pure convolutional structure. We demonstrate CSRNet on four datasets (ShanghaiTech dataset, the UCF_CC_50 dataset, the WorldEXPO'10 dataset, and the UCSD dataset) and we deliver the state-of-the-art performance. In the ShanghaiTech Part_B dataset, CSRNet achieves 47.3% lower Mean Absolute Error (MAE) than the previous state-of-the-art method. We extend the targeted applications for counting other objects, such as the vehicle in TRANCOS dataset. Results show that CSRNet significantly improves the output quality with 15.4% lower MAE than the previous state-of-the-art approach.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Domain Adaptation for Traffic Density Estimation (2021, methodology): “More recently, the authors of (Li et al., 2018) introduced CSRNet.”
Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework (2021, background): “They either conduct a pixel-wise density map regression [16, 28, 11, 1, 25, 8], or resort to classify the count value of local patch into several bins [39, 21, 22].”
JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method (2020, methodology): “Table 12 shows the results of the proposed method on ShanghaiTech as compared with several recent approaches: CP-CNN[6], IGCNN [30], D-ConvNet [33], Liu et al.[58], CSR-Net [75], icCNN [25], SA-Net[26], ACSCP [32] and Jian et al.[64], CANet [76], BCC [70], DSSI-Net [76], MBTTBF [63] and LSCCNN [69].”
NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization (2020, methodology): “Especially, benefiting from the development of deep learning in computer vision, the counting performance on the datasets [6], [7], [8], [9] is continuously refreshed by Convolutional Neural Networks (CNN)-based methods [10], [11], [12].”
Unsupervised vehicle counting via multiple camera domain adaptation (2020, neutral): “Concerning vehicle counting in urban spaces, where images are of very low resolution, and most objects are partially occluded, density-based methods have a clear advantage on detection methods [15, 6, 8, 3].”
Scale Aggregation Network for Accurate and Efficient Crowd Counting (2018, methodology): “[33] proposed CSRNet by combining VGG-16 [34] and dilated convolution layers to aggregate multi-scale contextual information.”
Attention Scaling for Crowd Counting (2020, neutral): “4 6 - - - - - - - 9 PACNN [31]+ [16] 62.”
Context-Aware Crowd Counting (2018, methodology): “The regressors can be based on Random Forests [18], Gaussian Processes [7], or more recently Deep Nets [41, 42, 26, 31, 40, 36, 32, 24, 19, 30, 33, 22, 15, 28, 5], with most state-of-the-art approaches now relying on the latter.”
ADCrowdNet: An Attention-Injective Deformable Convolutional Network for Crowd Understanding (2018, methodology): “We generate the ground truth for DME training following the procedure in [17].”
Efficient Attention Network: Accelerate Attention by Searching Where to Plug (2020, methodology): “To further investigate the generalization of EAN, we conduct experiments on transferring the optimal architecture from image classiﬁcation to crowd counting task [33, 4, 20, 12], semantic segmentation [9].”
Distribution Matching for Crowd Counting (2020, methodology): “Almost all prior crowd density map estimation methods [56, 57, 60, 38, 20, 33, 35, 4, 28, 50, 27, 40, 29, 26] have followed this convention.”
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks (2019, methodology): “In CSRNet [19], the density maps are simply generated by applying bilinear interpolation to up-sample the encoded feature maps.”
Incorporating Side Information by Adaptive Convolution (2017, methodology): “…prediction of CNN/ACNN; (2) we show that adaptive convolution is also applicable to deeper fully-convolutional baseline models, e.g., CSRNet (Li et al. 2018; 3) we conduct more thorough experiments on crowd counting, including results using FCN and AFCN, and results on WorldExpo’10 (Zhang…”
Where are the Blobs: Counting by Localization with Point Supervision (2018, neutral): “andom samples of the test set evaluation. Methods UCSD Mall ShanghaiTech B FCN-rLSTM [43] 1.54 - - MoCNN [45] - 2.75 - CNN-boosting [46] 1.10 2.01 - M-CNN [17] 1.07 - 26.4 CP-CNN [47] - - 20.1 CSRNet [44] 1.16 - 10.6 LC-FCN8 1.51 2.42 13.14 LC-ResFCN 0.99 2.12 25.89 LC-PSPNet 1.01 2.00 21.61 Table 4”
Learning Short-Cut Connections for Object Counting (2018, methodology): “[22] they extend the VGG16 with the dilated convolution [43] achieving the top performance of the state of the art.”
Deep Structural Contour Detection (2020, neutral): “[51], MCNN [52], Switch CNN [34], ACSCP [37], CSRnet [24], SA-Net [6], MAN [42] and RANet [50].”
Learning From Synthetic Data for Crowd Counting in the Wild (2019, methodology): “These scheme can effectively evaluated the algorithm on GCC. Table 2 reports the performance of our SFCN and two popular methods (MCNN [43] and CSRNet[19]) on the proposed GCC dataset.”
SDC – Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks (2019, background): “As others before [8, 22], we argue that convolution with dilation rate r and stride r is equal to convolution with dilation rate 1 (no dilation) of sub-sampled input by factor r (no smoothing).”
Object Counting and Instance Segmentation With Image-Level Supervision (2019, methodology): “Furthermore, we also compare with recent crowd counting approach (CSRnet) [16] on the person category of the PASCAL VOC 2007 by retraining it on the dataset.”