Soft Sampling for Robust Object Detection

By Zhe Wu, Navaneeth Bodla, Bharat Singh, Mahyar Najibi, Ramalingam Chellappa, Larry S. Davis. 2018

78 citations recorded by the source corpus.

We study the robustness of object detection under the presence of missing annotations. In this setting, the unlabeled object instances will be treated as background, which will generate an incorrect training signal for the detector. Interestingly, we observe that after dropping 30% of the annotations (and labeling them as background), the performance of CNN-based object detectors like Faster-RCNN only drops by 5% on the PASCAL VOC dataset. We provide a detailed explanation for this result. To further bridge the performance gap, we propose a simple yet effective solution, called Soft Sampling. Soft Sampling re-weights the gradients of RoIs as a function of overlap with positive instances. This ensures that the uncertain background regions are given a smaller weight compared to the hardnegatives. Extensive experiments on curated PASCAL VOC datasets demonstrate the effectiveness of the proposed Soft Sampling method at different annotation drop rates. Finally, we show that on OpenImagesV3, which is a real-world dataset with missing annotations, Soft Sampling outperforms standard detection baselines by over 3%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Step Toward More Inclusive People Annotations for Fairness (2021, background): “In particular, we do not yet understand the effects that structured patterns of the type we observe here have on model performance; typically studies of robustness in the presence of noise make simplifying assumptions and experiment with annotations that are modified or missing uniformly at random [”
EPicker is an exemplar-based continual learning approach for knowledge accumulation in cryoEM particle picking (2022, methodology): “Some previous works dealing with sparse annotation in natural images solved this problem by reweighting the importance of region proposals generated by the two-stage object detector Faster RCNN 38,39 or recalibrating the loss of the anchors used by the one-stage object detector YOLO 40 .”
Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNN (2020, methodology): “This also has been observed by (Wu et al., 2019b).”
Objects365: A Large-Scale, High-Quality Dataset for Object Detection (2019, unknown): “De-formable ConvNet [4, 39], Soft Sampling [34], SNIP [32], SNIPER [33], and Cascade R-CNN [1], DetNet [21] are introduced to further improve the performance.”
Detecting 11K Classes: Large Scale Object Detection Without Fine-Grained Bounding Boxes (2019, methodology): “Although researchers have developed several large scale algorithms, such as R-FCN 3000 [27] and SNIPER with soft sampling [37, 28] to facilitate detection on the scale of Ima-geNet and OpenImages detection set .”
Sampling Techniques for Large-Scale Object Detection From Sparsely Annotated Objects (2018, methodology): “Despite the idea of pseudo labels being widely recognized [26, 25], there is not much consensus on how to utilize it, especially for object detection.”
Solving Missing-Annotation Object Detection with Background Recalibration Loss (2020, methodology): “In the recent years, a variety of two-stage detector based methods have been proposed, such as part-aware sampling [6] and soft sampling [1].”
Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges (2021, result): “However, if the researchers want to study this problem and further improve the performance, we recommend researchers refer to these prior works [77], [78].”
Deep Regionlets for Object Detection (2017, background): “ wide adoption. Deformable Part Model-based detection (DPM) [12] proposed the concept of deformable part models to handle object deformations. Due to the rapid development of deep learning techniques [25, 20, 43, 5, 53, 37, 50, 2, 49], the deep learning-based detectors have become dominant object de”