Segment as Points for Efficient Online Multi-Object Tracking and Segmentation

By Zhenbo Xu, Wei Zhang, Xiao Tan, Wei Yang, Huan Huang, Shilei Wen, Errui Ding, Liusheng Huang. 2020

83 citations recorded by the source corpus.

Current multi-object tracking and segmentation (MOTS) methods follow the tracking-by-detection paradigm and adopt convolutions for feature extraction. However, as affected by the inherent receptive field, convolution based feature extraction inevitably mixes up the foreground features and the background features, resulting in ambiguities in the subsequent instance association. In this paper, we propose a highly effective method for learning instance embeddings based on segments by converting the compact image representation to un-ordered 2D point cloud representation. Our method generates a new tracking-by-points paradigm where discriminative instance embeddings are learned from randomly selected points rather than images. Furthermore, multiple informative data modalities are converted into point-wise representations to enrich point-wise features. The resulting online MOTS framework, named PointTrack, surpasses all the state-of-the-art methods including 3D tracking methods by large margins (5.4% higher MOTSA and 18 times faster over MOTSFusion) with the near real-time speed (22 FPS). Evaluations across three datasets demonstrate both the effectiveness and efficiency of our method. Moreover, based on the observation that current MOTS datasets lack crowded scenes, we build a more challenging MOTS dataset named APOLLO MOTS with higher instance density. Both APOLLO MOTS and our codes are publicly available at this https URL.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

  1. Tim Meinhardt
  2. L. Leal-Taix'e
  3. Alexander Kirillov
  4. Jiale Cao
  5. Christoph Feichtenhofer
  6. Junsong Yuan
  7. Ming Yang
  8. Yunchao Wei

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Yi Yang
  3. Xiaogang Wang
  4. Dacheng Tao
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Trevor Darrell
  8. Andrew Zisserman

In Environmental Engineering

  1. Yi Yang

Citations in the corpus