Repulsion Loss: Detecting Pedestrians in a Crowd

By Xinlong Wang, Tete Xiao, Yuning Jiang, Shuai Shao, Chunhua Shen. 2017

567 citations recorded by the source corpus.

Detecting individual pedestrians in a crowd remains a challenging problem since the pedestrians often gather together and occlude each other in real-world scenarios. In this paper, we first explore how a state-of-the-art pedestrian detector is harmed by crowd occlusion via experimentation, providing insights into the crowd occlusion problem. Then, we propose a novel bounding box regression loss specifically designed for crowd scenes, termed repulsion loss. This loss is driven by two motivations: the attraction by target, and the repulsion by other surrounding objects. The repulsion term prevents the proposal from shifting to surrounding objects thus leading to more crowd-robust localization. Our detector trained by repulsion loss outperforms the state-of-the-art methods with a significant improvement in occlusion cases.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, methodology): “We also migrate three image-level detection methods to the CMaskTrack R-CNN model, including 1) the repulsion loss [76] which requires the predicted boxes to keep away from other ground-truth boxes; 2) the compact loss [92] which enforces proposals to be close and locate compactly to the correspondi”
Acquisition of Localization Confidence for Accurate Object Detection (2018, background): “Meanwhile, Repulsion Loss is proposed in [29] to robustly detect objects with crowd occlusion.”
Detection in Crowded Scenes: One Proposal, Multiple Predictions (2020, background): “Previous works have tried to address this issue from different perspectives, such as sophisticated NMS [1, 16, 23, 18, 25, 17], new loss functions [44, 40], re-scoring [19], part-based detectors [44, 38, 46, 3].”
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge (2021, background): “[34, 41] propose new loss functions to enforce predicted box to locate compactly to the corresponding ground-truth objects while far from other objects.”
Recent Advances in Deep Learning for Object Detection (2019, background): “[213] proposed a novel Repulsion Loss to detect pedestrians in a crowd.”
MetaAnchor: Learning to Detect Objects with Customized Anchors (2018, background): “via clustering [31], which is especially critical on specific detection tasks such as anchor-based face [40, 45, 28, 36, 43] and pedestrian [41, 3, 44, 26] detections.”
CrowdHuman: A Benchmark for Detecting Human in a Crowd (2018, methodology): “ [9] propose a learning method to improve the robustness of NMS. Part-based models are utilized in [20,33] to alleviate occlusion problem. Repulsion loss is proposed to detect persons in crowd scenes [26]. 3. CrowdHuman Dataset In this section, we describe our CrowdHuman dataset including the collec”
Occlusion-aware R-CNN: Detecting Pedestrians in a Crowd (2018, methodology): “[55] design a repulsion loss, which not only pushes each proposal to approach its designated target, but also to keep it away from the other ground truth objects and their corresponding designated proposals.”
Adaptive NMS: Refining Pedestrian Detection in a Crowd (2019, methodology): “To address this problem, [44] and [49] propose additional penalties to produce more compact bounding boxes and thus become less sensitive to the threshold of NMS.”
Object Detection in 20 Years: A Survey (2019, background): “To this end, some researchers have designed new loss function by considering the attraction of target and the repulsion of other surrounding objects [318].”
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems (2020, methodology): “ile maintaining the fastest speed. 2 Relate Work 2.1 Multispectral Pedestrian Detection CNN-based pedestrian detection has achieved notable progress in recent years with methods of occlusion handling [46,32,38], cascaded detection systems [30,2], semantic attention [49,3], anchor-free approach [31],”
KGSNet: Key-Point-Guided Super-Resolution Network for Pedestrian Detection in the Wild (2020, methodology): “…heavy evaluation protocols, we compare our proposed KGSNet approach with some state-of-the-art methods: DeepParts [28], F-DNN+SS [20], FasterRCNN+ATT [40], CompACT-Deep [78], SA-FastRCNN [61], RPN+BF [22], MS-CNN [79], GDFL [39], TLL-TFA [41], SDS-RCNN [19], PDOE [38], RepLoss [25] and ALFNet [63].”
M3D-RPN: Monocular 3D Region Proposal Network for Object Detection (2019, methodology): “We use a negative logistic loss applied to the IoU between the matched ground truth box b̂2D and the transformed b2D for Lb2D , similar to [37, 43], defined as:”
High-Level Semantic Feature Detection: A New Perspective for Pedestrian Detection (2019, methodology): “4 (c) shows that CSP also performs very well for heavily occluded pedestrians, outperforming RepLoss [46] and OR-CNN [52] which are explicitly designed for occlusion cases.”
Set Prediction without Imposing Structure as Conditional Density Estimation (2020, methodology): “In this work we focus on distributions taking the form of deep energy-based models (Ngiam et al., 2011; Zhai et al., 2016; Belanger & McCallum, 2016): with Z as the partition function and E θ the energy function with parameters θ .”
Learning Instance Occlusion for Panoptic Segmentation (2019, methodology): “A repulsion loss is added to a pedestrian detection algorithm [37] to deal with the crowd occlusion problem, but it focuses on detection only, without instance segmentation.”
Occluded Prohibited Items Detection: An X-ray Security Inspection Benchmark and De-occlusion Attention Module (2020, background): “There are several works trying to solve occlusion problems in different scenarios, such as person re-identification [37, 42, 43], face recognition [6, 28, 38].”
Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection (2018, methodology): “Recent work [32, 33] were built upon Faster R-CNN architecture and focused on occlusion handling.”
Learning Efficient Single-Stage Pedestrian Detectors by Asymptotic Localization Fitting (2018, methodology): “Our method achieves MR−2 of 4.5 under the IoU threshold of 0.5, which is comparable to the best competitor (4.0 of RepLoss [40]) .”