Bottom-Up Object Detection by Grouping Extreme and Center Points

By Xingyi Zhou, Jiacheng Zhuo, Philipp Krahenbuhl. 2019

927 citations recorded by the source corpus.

With the advent of deep learning, object detection drifted from a bottom-up to a top-down recognition problem. State of the art algorithms enumerate a near-exhaustive list of object locations and classify each into: object or not. In this paper, we show that bottom-up approaches still perform competitively. We detect four extreme points (top-most, left-most, bottom-most, right-most) and one center point of objects using a standard keypoint estimation network. We group the five keypoints into a bounding box if they are geometrically aligned. Object detection is then a purely appearance-based keypoint estimation problem, without region classification or implicit feature learning. The proposed method performs on-par with the state-of-the-art region based detection methods, with a bounding box AP of 43.7% on COCO test-dev. In addition, our estimated extreme points directly span a coarse octagonal mask, with a COCO Mask AP of 18.9%, much better than the Mask AP of vanilla bounding boxes. Extreme point guided segmentation further improves this to 34.6% Mask AP.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection (2019, methodology): “anchor-free keypoint-based: ExtremeNet [71] trainval35k Hourglass-104 40.”
Objects as Points (2019, methodology): “1 FPS) and ExtremeNet [61]( 40 .”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, methodology): “Using keypoint detection to predict eight extreme points of one instance and generate an octagon mask, Ex-tremeNet (Zhou, Zhuo, and Krahenbuhl 2019) can achieve relatively reasonable object mask prediction.”
Incremental Few-Shot Object Detection (2020, neutral): “It is inspired by keypoint detection methods [33, 52], taking a similar spirit to [25, 57] without the need for point grouping and postprocessing.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “ExtremeNet [79] groups ‘extreme points’ [57] according to the relation to a center point.”
CenterNet: Keypoint Triplets for Object Detection (2019, methodology): “keypoint-based object detection approaches [43, 29, 50, 49] are proposed to avoid the disadvantages of the use of anchor boxes and bounding box regression.”
Conditional DETR for Fast Training Convergence (2021, methodology): “Typical methods include YOLOv1 [30], CornerNet [21], ExtremeNet [50], CenterNet [49, 6], FCOS [39] and others [23, 28, 52, 19, 51, 22, 15, 46, 47].”
Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering (2019, background): “Even “one-stage” object detection approaches, where these boxes become category-speciﬁc, can be formulated in a bottom-up manner as detecting and grouping extreme and center points (Zhou et al., 2019).”
Energy-Based Models for Deep Probabilistic Regression (2020, background): “human pose estimation [5, 57, 52] and object detection [28, 63], where a 2D heatmap over image pixel coordinates y is predicted.”
SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing (2020, background): “FCOS [31], CornerNet [32], CenterNet [33] and ExtremeNet [34] attempt to predict some keypoints of objects such as corners or extreme points, which are then grouped into bounding boxes, and these detectors have also been applied to the field of remote sensing [35], [36].”
CentripetalNet: Pursuing High-Quality Keypoint Pairs for Object Detection (2020, background): “0 ExtremeNet(multi-scale) [42] Hourglass-104 43.”
Probabilistic Anchor Assignment with IoU Prediction for Object Detection (2020, background): “These models predict various key points such as corners [18], extreme points [44], center points [6,33] or arbitrary feature points [38] induced from deformable convolution [5].”
A Survey of Deep Learning-Based Object Detection (2019, background): “To address that, recently propose a series of anchor-free methods [61] [62] [49] [108] [109] [110] [111] [112] [113]”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, background): “Alternatively, several single-stage approaches [26, 22, 52, 50, 12] propose an anchor box free strategy that eliminates the need for anchor boxes.”
Rethinking Classification and Localization for Object Detection (2019, methodology): “Point-based methods [21, 22, 47, 7, 48] model an object as keypoints (corner, center, etc), and are built on keypoint estimation networks.”
Imbalance Problems in Object Detection: A Review (2019, methodology): “Bottom-up detectors [23], [48], [49] adopt a completely different approach to object detection than the one-stage and the two-stage detectors (see Section 2.”
Revisiting the Sibling Head in Object Detector (2020, neutral): “9 ExtremeNet [42] Hourglass-104 [28] 40.”