Switching Convolutional Neural Network for Crowd Counting

By Deepak Babu Sam, Shiv Surya, R. Venkatesh Babu. Computer Vision and Pattern Recognition, 2017

964 citations recorded by the source corpus.

We propose a novel crowd counting model that maps a given crowd scene to its density. Crowd analysis is compounded by myriad of factors like inter-occlusion between people due to extreme crowding, high similarity of appearance between people and background elements, and large variability of camera view-points. Current state-of-the art approaches tackle these factors by using multi-scale CNN architectures, recurrent networks and late fusion of features from multi-column CNN with different receptive fields. We propose switching convolutional neural network that leverages variation of crowd density within an image to improve the accuracy and localization of the predicted crowd count. Patches from a grid within a crowd scene are relayed to independent CNN regressors based on crowd count prediction quality of the CNN established during training. The independent CNN regressors are designed to have different receptive fields and a switch classifier is trained to relay the crowd scene patch to the best CNN regressor. We perform extensive experiments on all major crowd counting datasets and evidence better performance compared to current state-of-the-art methods. We provide interpretable representations of the multichotomy of space of crowd scene patches inferred from the switch. It is observed that the switch relays an image patch to a particular CNN column based on density of crowd.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/9adcf858bda2991627951b68b75c99fc1ebd0f76

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

Attention Scaling for Crowd Counting (2020, methodology): “[29] proposed Switch-CNN that utilizes a switch classifier to choose the optimal one from the density generator pool.”
Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey (2019, background): “…Horn et al. 2018), X-ray scans (Rajpurkar et al. 2017) or galaxies (Dieleman et al. 2015)], image segmentation [with applications to crowd counting (Sam et al. 2017), autonomous driving (Siam et al. 2017) or satellite imagery (Mnih 2013)], along with many other miscellaneous applications Table 1…”
Crowd Counting via Adversarial Cross-Scale Consistency Pursuit (2018, methodology): “Recent CNN-based works [20, 37, 25] utilize multi-path architectures to address the scale variation issue and have achieved good improvements in crowd density estimation.”
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks (2019, methodology): “Moreover, we also outperform MCNN on all datasets in terms of MAE and MSE.”
Adaptive Weighting Multi-Field-Of-View CNN for Semantic Segmentation in Pathology (2019, methodology): “For these experiments, we compared the segmentation accuracy with the following state-of-the-art methods: U-net [25][36], SegNet [3][24], Dilated-net [46], DeepLabv3+ [9][23], and Hard-Switch-CNN (HS) [37] that adaptively selects an expert network (does not aggregate multiple images).”
Iterative Crowd Counting (2018, methodology): “Earlier works [20,15] have tackled this challenge by using a multi-column or a switching CNN architecture.”
Distribution Matching for Crowd Counting (2020, methodology): “Additionally, even without using a multi-scale architecture as in [4, 47], or a deeper network as in [2, 50], DM-Count still achieves state-of-the-art performance on all four datasets.”
DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation (2017, methodology): “Better performance can be obtained by further exploiting switching structures [31, 28, 17] or contextual correlations using LSTM [29].”
CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes (2018, methodology): “Also, we choose the Euclidean distance to measure the difference between the ground truth and the estimated density map we generated which is similar to other works [19, 18, 4].”
Scale Aggregation Network for Accurate and Efficient Crowd Counting (2018, methodology): “Method MCNN [3] Switch-CNN [5] CP-CNN [6] CSRNet [33] SANet”
Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds (2018, methodology): “SwitchCNN [24] uses a similar three column network; however, it also employs a switching network that decides which column should exclusively handle the input patch.”
Learning Short-Cut Connections for Object Counting (2018, methodology): “The size of the filter is chosen to be [4,4] for those features maps that are greater or equal to [24,24], while for those that are smaller we use a slightly reduced size of [3,3].”
Bayesian Loss for Crowd Count Estimation With Point Supervision (2019, neutral): “4 SWITCH-CNN [3] 228 445 90.”
Learning From Synthetic Data for Crowd Counting in the Wild (2019, background): “In addition, we find the results on Shanghai Tech B achieve a good level, even outperforms some early supervised methods [43, 35, 31, 36, 20].”
Context-Aware Crowd Counting (2018, methodology): “Previous works in crowd density estimation use the mean absolute error (MAE) and the root mean squared error (RMSE) as evaluation metrics [42, 41, 26, 31, 40, 36].”
JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method (2020, methodology): “The proposed method is compared with the following recent methods: Idrees et al.[3], MCNN [5], CMTL [24], SwitchingCNN [7], Idrees et al.[21], Jian et al.[64], CA-Net [76], BCC [70], DSSI-Net [76], MBTTBF [63] and LSC-CNN [69].”
Incorporating Side Information by Adaptive Convolution (2017, methodology): “Hence, in contrast to Zhang et al. (2015, 2016), Onoro-Rubio and López-Sastre (2016), Sam et al. (2017) and Sindagi and Patel (2017), we propose to use the existing side information (e.g. perspective weight) as an input to adapt the convolutional layers to different scenes.”
NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization (2020, background): “Especially, benefiting from the development of deep learning in computer vision, the counting performance on the datasets [6], [7], [8], [9] is continuously refreshed by Convolutional Neural Networks (CNN)-based methods [10], [11], [12], [13], [14].”