Generating High-Quality Crowd Density Maps Using Contextual Pyramid CNNs

By Vishwanath A. Sindagi, Vishal M. Patel. IEEE International Conference on Computer Vision, 2017

669 citations recorded by the source corpus.

We present a novel method called Contextual Pyramid CNN (CP-CNN) for generating high-quality crowd density and count estimation by explicitly incorporating global and local contextual information of crowd images. The proposed CP-CNN consists of four modules: Global Context Estimator (GCE), Local Context Estimator (LCE), Density Map Estimator (DME) and a Fusion-CNN (F-CNN). GCE is a VGG-16 based CNN that encodes global context and it is trained to classify input images into different density classes, whereas LCE is another CNN that encodes local context information and it is trained to perform patch-wise classification of input images into different density classes. DME is a multi-column architecture-based CNN that aims to generate high-dimensional feature maps from the input image which are fused with the contextual information estimated by GCE and LCE using F-CNN. To generate high resolution and high-quality density maps, F-CNN uses a set of convolutional and fractionally-strided convolutional layers and it is trained along with the DME in an end-to-end fashion using a combination of adversarial loss and pixellevel Euclidean loss. Extensive experiments on highly challenging datasets show that the proposed method achieves significant improvements over the state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/0059525922079e1ff112139e01f44b8e2d69264b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation (2017, methodology): “Specifically, the ensemble and fusion strategy is employed by the M-CNN [42], Switching-CNN [28], CP-CNN [31] in Table 2.”
Crowd Counting via Adversarial Cross-Scale Consistency Pursuit (2018, methodology): “The latest research CP-CNN [28] proposes a contextual Pyramid CNNs for incorporating global and local contexts which are obtained by learning various density levels.”
Learning Multi-Level Density Maps for Crowd Counting (2020, methodology): “Furthermore, Sindagi and Patel [25] incorporated global and local contextual information of crowd images to generate high-quality crowd density maps.”
NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization (2020, background): “tance of crowd counting, many researchers [3], [4], [5] pay attention to it and achieve quite a few significant improvements in this field.”
Drone-Based Object Counting by Spatially Regularized Regional Proposal Network (2017, positive): “[26] incorporate global and lo-cal contextual information for better estimating the crowd counts.”
Density-Aware Single Image De-raining Using a Multi-stream Dense Network (2018, background): “leveraged in various applications such as semantic segmentation [45], face-alignment [22], visual tracking [18] crowdcounting [30], single image super-resolution[43], face antiSpoofing [1], action recognition [48], depth estimation [5], single image dehazing [24, 42, 40] and also in single image de-”
JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method (2020, methodology): “Consequently, computer vision-based crowd analytics and surveillance [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] have received increased interest.”
CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes (2018, methodology): “[5] present a Contextual Pyramid CNN, which uses CNN networks to estimate context at various levels for achieving lower count error and better quality density maps.”
Multi-style Generative Network for Real-time Transfer (2017, background): “ning. The Generative Adversarial Network (GAN) [17], which jointly trains an adversarial generator and discriminator simultaneously, has catalyzed a surge of interest in the study of image generation [2,22,23,36,40,44,45]. Recent work on imageto-image GAN [23] adopts a conditional GAN to provide a g”
Densely Connected Pyramid Dehazing Network (2018, background): “The success of GANs in synthesizing realistic images has led researchers to explore the adversarial loss for various low-level vision applications such as text-to-image synthesis[32, 52, 54, 6], image-image translation [18, 28, 46] and other applications [23, 50, 55, 58, 45, 38, 44, 31].”
Context-Aware Crowd Counting (2018, methodology): “In [36], this was done by learning to predict pre-defined density levels.”
Iterative Crowd Counting (2018, methodology): “On Part-B of the Shanghaitech dataset, using the one-stage ic-CNN which has a simpler architecture than the ﬁve-branch CP-CNN [16], we improve on the previously reported state of the art results by 48 .”
Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds (2018, background): “Sindagi and Patel [26] presented a CNNbased approach that incorporates global and local contextual information in an image to generate density maps.”
Learning Short-Cut Connections for Object Counting (2018, methodology): “Note that both CP-CNN and CSRNet approaches are based on or include the VGG16 architecture.”
Attention Scaling for Crowd Counting (2020, methodology): “These methods have tried to exploit multi-scale feature fusion [49, 29, 35], multi-task learning [34, 30, 21], and the attention mechanism [45, 44, 19] to solve the above questions.”
Scale Aggregation Network for Accurate and Efficient Crowd Counting (2018, methodology): “[32,6] explored methods to incorporate the contextual information by learning various density levels and generate high-resolution density maps.”