Spatial Semantic Regularisation for Large Scale Object Detection

By Damian Mrowca, Marcus Rohrbach, Judy Hoffman, Ronghang Hu, Kate Saenko, Trevor Darrell. 2015

25 citations recorded by the source corpus.

Large scale object detection with thousands of classes introduces the problem of many contradicting false positive detections, which have to be suppressed. Class-independent non-maximum suppression has traditionally been used for this step, but it does not scale well as the number of classes grows. Traditional non-maximum suppression does not consider label-and instance-level relationships nor does it allow an exploitation of the spatial layout of detection proposals. We propose a new multi-class spatial semantic regularisation method based on affinity propagation clustering, which simultaneously optimises across all categories and all proposed locations in the image, to improve both the localisation and categorisation of selected detection proposals. Constraints are shared across the labels through the semantic WordNet hierarchy. Our approach proves to be especially useful in large scale settings with thousands of classes, where spatial and semantic interactions are very frequent and only weakly supervised detectors can be built due to a lack of bounding box annotations. Detection experiments are conducted on the ImageNet and COCO dataset, and in settings with thousands of detected categories. Our method provides a significant precision improvement by reducing false positives, while simultaneously improving the recall.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiaogang Wang
2. Wanli Ouyang
3. Hongsheng Li
4. Larry S. Davis
5. Junjie Yan
6. Bharat Singh
7. Ramalingam Chellappa
8. Bernt Schiele

In Computer Vision and Pattern Recognition

1. Xiaogang Wang
2. Luc Van Gool
3. Wanli Ouyang
4. Hongsheng Li
5. Yi Yang
6. Dacheng Tao
7. Andrew Zisserman
8. Andrea Vedaldi

In Media Technology

1. Wanli Ouyang
2. Junjie Yan
3. Andrea Vedaldi
4. Larry S. Davis
5. Tong Xiao
6. Bharat Singh
7. Kai Kang
8. Diane Larlus

Citations in the corpus

• Learning Non-maximum Suppression (2017, neutral): “Several other clustering algorithms have been explored for the task of NMS without showing consistent gains: mean-shift clustering [6, 35], agglomerative clustering [2], affinity propagation clustering [17], and heuristic variants [25].”
• Soft-NMS — Improving Object Detection with One Line of Code (2017, methodology): “A few learning-based meth ds have been p oposed as an al ernative to greedy NMS hich obtain good performance for object class det ctio [5, 26, 21].”
• I Have Seen Enough: Transferring Parts Across Categories (2016, methodology): “Fewer works consider transfer learning for localization as we do; these include the works of [9, 19, 28, 33] that transfer bounding box information using the ImageNet hierarchies; the method of [22] that transfer object detectors from seed classes; and [1] which transfers detectors assuming a limite”
• T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos (2016, background): “State-of-the-art methods for detecting objects of general classes are mainly based on deep CNNs [1], [3], [4], [5], [8], [10], [11], [12], [13], [14], [15], [16], [17].”