Unsupervised Hard Example Mining from Videos for Improved Object Detection
By SouYoung Jin, Aruni RoyChowdhury, Huaizu Jiang, Ashish Singh, Aditya Prasad, Deep Chakraborty, Eric Learned-Miller. 2018
69 citations recorded by the source corpus.
Important gains have recently been obtained in object detection by using training objectives that focus on hard negative examples, i.e., negative examples that are currently rated as positive or ambiguous by the detector. These examples can strongly influence parameters when the network is trained to correct them. Unfortunately, they are often sparse in the training data, and are expensive to obtain. In this work, we show how large numbers of hard negatives can be obtained automatically by analyzing the output of a trained detector on video sequences. In particular, detections that are isolated in time, i.e., that have no associated preceding or following detections, are likely to be hard negatives. We describe simple procedures for mining large numbers of such hard negatives (and also hard positives) from unlabeled video data. Our experiments show that retraining detectors on these automatically obtained examples often significantly improves performance. We present experiments on multiple architectures and multiple data sets, including face detection, pedestrian detection and other object categories.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ludwig Schmidt
- B. Recht
- Vaishaal Shankar
- Quoc V. Le
- Deva Ramanan
- Achal Dave
- Percy Liang
- Rebecca Roelofs
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Andrew Zisserman
- Xiaogang Wang
- Trevor Darrell
- Andrea Vedaldi
- Jitendra Malik
Citations in the corpus
- Do Image Classifiers Generalize Across Time? (2019, methodology): “Authors of [16] explicitly identify flickering failures and use a technique reminiscent of adversarially robust training to improve image-based models.”
- A Survey on Curriculum Learning (2021, background): “There is also a line of research named hard example mining (HEM) [45], [101] selecting the most difficult examples in each training batch.”
- A systematic framework for natural perturbations from videos (2019, methodology): “[14] explicitly identify such failures, and use a technique reminiscent of adversarially robust training to improve image-based models.”
- Automatic Adaptation of Object Detectors to New Domains Using Self-Training (2019, methodology): “(a) Baseline; (b) HP [25]; (c) Ours; (d) DA[8].”
- Contrastive Learning with Hard Negative Samples (2020, background): “[24] exploit the specific temporal structure of video to generate negatives for object detection.”