Center-based 3D Object Detection and Tracking

By Tianwei Yin, Xingyi Zhou, Philipp Krahenbuhl. Computer Vision and Pattern Recognition, 2020

2,302 citations recorded by the source corpus.

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, Center-Point outperforms all previous single model methods by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/22d40963e633e1b4af4a9fefda68e1b8dc96ba63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, methodology): “…lie in two categories based on their feature modality: i) point-based methods that directly operate on the raw LiDAR point clouds [38, 37, 36, 42, 57, 22]; and ii) transforming the original point clouds into a Euclidean feature space, such as 3D voxels [65] and feature pillar [20, 52, 59].”
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, neutral): “3D object detection is often performed in LiDAR point clouds [55, 18, 51, 52, 34, 36, 10, 39].”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, methodology): “Following CenterPoint [55], the ground-truths of instances are randomly rotated with a range of [ − 22 .”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, methodology): “Following Center-Point [58], our FFN predicts the center offset from the query position as δx, δy , bounding box height as z , size l, w, h as log( l ) , log( w ) , log( h ) , yaw angle α as sin ( α ) , cos ( α ) and the velocity (if available) as v x , v y .”
nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles (2021, methodology): “We use PointPillars [12] with CenterPoint [23], a modiﬁed version multi-view fusion (MVF++) [17], and non-causal tracking to achieve near-human labeling performance.”
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, neutral): “463 - - - - CenterPoint-Voxel [51] L - 0.”
Embracing Single Stride 3D Object Detector with Sparse Transformer (2021, methodology): “Some previous works [14, 16, 65] in 3D detection have found that the performance can ben-eﬁt from the recovery of output resolution by upsampling.”
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection (2021, methodology): “We use the LiDAR-only detector Center-Point [27] as baseline.”
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, background): “Although two-stage detectors are common in LiDAR-based 3D object detection [1, 5, 12, 16, 28, 38, 41], their application in camera-based 3D detection is far less well known.”
3D Object Detection for Autonomous Driving: A Survey (2021, background): “Voxel based Methods Works in this group voxelize irregular point clouds to 2D/3D compact grids and then collapse it to a bird’s-eyeview 2D representation on which CNNs effectively convolve against [76, 48, 49, 77, 23, 24, 25, 10, 6, 26, 27, 28, 29, 5, 56, 57, 55].”
One Million Scenes for Autonomous Driving: ONCE Dataset (2021, methodology): “Many techniques have been explored for 3D object detection in driving scenarios, and they can be broadly categorized into two classes: 1) Single-modality 3D detectors [49, 36, 24, 37, 52, 57, 35, 50] are designed to detect objects solely from sparse point clouds.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “For multi-frame setting with n sweeps, we follow previous work [24] and attach all point clouds together with relative time offsets to formulate the input XP .”
BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection (2022, methodology): “Thus, the existing paradigms [40, 41, 42, 22, 15] perform relatively poorly in predicting the time-relevant targets like velocity than the LiDAR-based [46] or radar-based [27] methods.”
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021, methodology): “For 3D detection, we employ the recently-proposed CenterPoint architecture [41].”
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “Furthermore, StreamPETR ex-cels over CenterPoint [54] in AMOTA, and demonstrates superior benefits in RECALL.”
Point2Seq: Detecting 3D Objects as Sequences (2022, methodology): “Anchor-based methods [13, 38] place predeﬁned anchors on each pixel center of a Bird-Eye-View (BEV) feature map, while center-based approaches [9, 42] treat the pixels near object centers as positives and predict boxes using those pixel features.”
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (2022, background): “Pre-aligned 3D visual understanding research [6, 12, 16, 17, 24, 25] is drawing significant attention in recent years due to the increasing demand of real-world applications such as augmented/virtual reality [1, 27, 31, 46], autonomous driving [23, 57] and robotics [2, 48].”