RepPoints: Point Set Representation for Object Detection

By Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin. 2019

1,092 citations recorded by the source corpus.

Modern object detectors rely heavily on rectangular bounding boxes, such as anchors, proposals and the final predictions, to represent objects at various recognition stages. The bounding box is convenient to use but provides only a coarse localization of objects and leads to a correspondingly coarse extraction of object features. In this paper, we present \textbf{RepPoints} (representative points), a new finer representation of objects as a set of sample points useful for both localization and recognition. Given ground truth localization and recognition targets for training, RepPoints learn to automatically arrange themselves in a manner that bounds the spatial extent of an object and indicates semantically significant local areas. They furthermore do not require the use of anchors to sample a space of bounding boxes. We show that an anchor-free object detector based on RepPoints can be as effective as the state-of-the-art anchor-based detection methods, with 46.5 AP and 67.4 $AP_{50}$ on the COCO test-dev detection benchmark, using ResNet-101 model. Code is available at \href{https://github.com/microsoft/RepPoints}{\color{cyan}{https://github.com/microsoft/RepPoints}}.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, background): “RepPoints [65] represents objects as a set of sample points and learns to arrange themselves in a manner that bounds the spatial extent of an object and indicates semantically signiﬁcant local areas.”
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection (2020, neutral): “7 ICCV19 RepPoints [30] R-101-DCN 24 X 11.”
End-to-End Semi-Supervised Object Detection with Soft Teacher (2021, methodology): “There are two mainstreams: single-stage object detectors [17, 21, 30] and two-stage object detectors [6, 22, 14, 34, 35].”
Dynamic Head: Unifying Object Detection Heads with Attentions (2021, methodology): “We ﬁnd recent studies [12, 35, 28, 31, 33] only focus on solving one of the aforementioned problems in various ways.”
YOLOv4: Optimal Speed and Accuracy of Object Detection (2020, background): “It is also possible to make a twostage object detector an anchor-free object detector, such as RepPoints [87].”
Focal Self-attention for Local-Global Interactions in Vision Transformers (2021, methodology): “In Table 3 To further verify the effectiveness of our proposed Focal Transformers, we follow [44] to train four different object detectors including Cascade R-CNN [7], ATSS [81], RepPoints [72] and Sparse R-CNN [55].”
VarifocalNet: An IoU-aware Dense Object Detector (2020, methodology): “To verify the generality of our varifocal loss, we apply it to some existing popular dense object detectors, including RetinaNet [8], FoveaBox [15], RepPoints [24] and ATSS [12], and evaluate the performance on the val2017.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “On the other hand, studies of the anchor-free direction mainly fall into keypoints-based [32], [33], [34], [35] and center-based [36], [37], [38], [39] paradigms.”
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios (2021, neutral): “4) anchor-free detectors: CenterNet [63], YOLOX [11], RepPoints [55].”
Probabilistic two-stage detection (2021, methodology): “RepPoint (Yang et al., 2019) and DenseRepPoint (Yang et al.”
3DSSD: Point-Based 3D Single Stage Object Detector (2020, background): “In 2D single-stage detectors, intersection-over-union (IoU) [15] threshold or mask [25, 30] can be used.”
Rethinking Transformer-based Set Prediction for Object Detection (2020, methodology): “We compare TSP-RCNN with multiple one-stage and two-stage object detection models [26, 31, 1, 28, 19, 40, 30, 3, 16, 35, 38] that also use ResNet-101 backbone or its deformable convolution network (DCN) [41] variant in Table 4.”
Learning to Match Anchors for Visual Object Detection (2021, methodology): “To improve adaptability, RepPoint [32] learned to automatically arrange keypoints in a manner which bounds the spatial extent of an object and indicates semantically signiﬁcant local areas.”
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation (2020, background): “Different to [51,50] that aim to learn accurate geometric localization, our approach aims to generate better features for box classification and coefficient generation.”
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, methodology): “Our detection branch is built on top of CenterNet [10] but other anchor-free methods such as [16], [18], [47], [48] can also be used.”