Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds

By Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Ali Al-Máadeed, Nasir M. Rajpoot, Mubarak Shah. European Conference on Computer Vision, 2018

831 citations recorded by the source corpus.

With multiple crowd gatherings of millions of people every year in events ranging from pilgrimages to protests, concerts to marathons, and festivals to funerals; visual crowd analysis is emerging as a new frontier in computer vision. In particular, counting in highly dense crowds is a challenging problem with far-reaching applicability in crowd safety and management, as well as gauging political significance of protests and demonstrations. In this paper, we propose a novel approach that simultaneously solves the problems of counting, density map estimation and localization of people in a given dense crowd image. Our formulation is based on an important observation that the three problems are inherently related to each other making the loss function for optimizing a deep CNN decomposable. Since localization requires high-quality images and annotations, we introduce UCF-QNRF dataset that overcomes the shortcomings of previous datasets, and contains 1.25 million humans manually marked with dot annotations. Finally, we present evaluation measures and comparison with recent deep CNNs, including those developed specifically for crowd counting. Our approach significantly outperforms state-of-the-art on the new dataset, which is the most challenging dataset with the largest number of crowd annotations in the most diverse set of scenes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bfdc7cde3bbfcba738a5eefe9143417ebf7d8f5c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bayesian Loss for Crowd Count Estimation With Point Supervision (2019, methodology): “[16] proposed a composition loss, which consists of (cid:96) 1 , (cid:96) 2 , and ∞ norm losses for the density map and a count regression loss.”
Synthetic Data for Deep Learning (2019, background): “a single surveillance camera, the ShanghaiTech dataset [697] has about 330,000 heads but only about 1200 images, again collected on the same event, and the UCF-QNRF dataset [270], while more diverse than previous ones, is limited to extremely congested scenes, with up to 12,000 people on the same im”
Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework (2021, methodology): “Different from the greedy association used in [10, 30], we apply a sequential association in which those higher scored predictions are associated firstly.”
Adaptive NMS: Refining Pedestrian Detection in a Crowd (2019, methodology): “Therefore, different from them, we propose to learn the density around each ground truth object as its own suppression threshold, sharing some similarity with the crowd density map estimation in the people counting task [16, 29].”
Attention Scaling for Crowd Counting (2020, methodology): “Even so, there is still much room for improvement in counting performance, especially in several challenging crowd datasets [49, 10, 11].”
Context-Aware Crowd Counting (2018, methodology): “They are those of [5], [15], [5], and [32], respectively.”
Distribution Matching for Crowd Counting (2020, background): “Most recent works [20, 35, 15, 4, 31, 50, 27, 40, 29, 26, 54, 8, 25, 55, 61, 59, 17, 48, 21, 23, 30, 47, 53, 43, 37, 56, 18, 39, 24, 44, 42] are based on density map estimation, which has been shown to be more robust than detection-then-count and count regression approaches.”
Deep Structural Contour Detection (2020, methodology): “Our MAE result of Part A dataset also achieves the second-best result, which is close to Figure 5: The qualitative results of the SOTA comparisons on the UCF-QNRF dataset.”
NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization (2020, methodology): “For further research on the extremely congested crowd counting,UCF-QNRF [9] is presented by Idrees et al. It is composed of 1,525 images with more than 1,251,642 label points.”
JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method (2020, methodology): “The proposed method is compared with the following recent methods: Idrees et al.[3], MCNN [5], CMTL [24], SwitchingCNN [7], Idrees et al.[21], Jian et al.[64], CA-Net [76], BCC [70], DSSI-Net [76], MBTTBF [63] and LSC-CNN [69].”
Learning From Synthetic Data for Crowd Counting in the Wild (2019, methodology): “[15] propose a large-scale extremely congested dataset.”