Multi-Object Tracking Through Simultaneous Long Occlusions and Split-Merge Conditions

By A. Perera, C. Srinivas, A. Hoogs, G. Brooksby, Wensheng Hu. Computer Vision and Pattern Recognition, 2006

314 citations recorded by the source corpus.

A fundamental requirement for effective automated analysis of object behavior and interactions in video is that each object must be consistently identified over time. This is difficult when the objects are often occluded for long periods: nearly all tracking algorithms will terminate a track with loss of identity on a long gap. The problem is further confounded by objects in close proximity, tracking failures due to shadows, etc. Recently, some work has been done to address these issues using higher level reasoning, by linking tracks from multiple objects over long gaps. However, these efforts have assumed a one-to-one correspondence between tracks on either side of the gap. This is often not true in real scenarios of interest, where the objects are closely spaced and dynamically occlude each other, causing trackers to merge objects into single tracks. In this paper, we show how to efficiently handle splitting and merging during track linking. Moreover, we show that we can maintain the identities of objects that merge together and subsequently split. This enables the identity of objects to be maintained throughout long sequences with difficult conditions. We demonstrate our approach on a highly challenging, oblique-view video sequence of dense traffic of a highway interchange. We successfully track the large majority of the hundreds of moving vehicles in the scene, many in close proximity, through long occlusions and shadows.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8b5f78b504958de5f660724411409033f037e53b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Multiobject tracking as maximum weight independent set (2011, background): “Brute-force strategies have been proposed to handle errors in the track linking by augmenting the initial set of tracks with their merges and splits [19].”
Seq-NMS for Video Object Detection (2016, methodology): “With regards to the association step, a classic method involved using Kalman filters to predict tracks and the Hungarian method [8, 15] to associate detections between frames.”
Learning to associate: HybridBoosted multi-target tracker for crowded scene (2009, methodology): “[4][5][6][7][8] aim at simultaneously optimizing all tracks in polynomial time by tailoring various optimization algorithms such as Linear Programming [4], Quadratic Boolean Programming [5], min-cost flow [6] and Hungarian algorithm [7] or its hierarchical version [8].”
Simple online and realtime tracking (2016, methodology): “When considering only one-to-one correspondences modelled as bi-partite graph matching, globally optimal solutions such as the Hungarian algorithm [15] can be used [10, 20].”
Globally-optimal greedy algorithms for tracking a variable number of objects (2011, methodology): “In practice, the typical solution is to apply non-max suppression (NMS) as a pre-process to prune the set of candidate locations V prior to multi-object tracking [19, 6, 14, 25].”
Multi-target tracking by learning local-to-global trajectory models (2015, methodology): “Given only the image information in a small time window, the local association methods are difficult to tackle the long-term occlusion due to the ambiguous and noisy observations, which make them incline to result in tracking failures (e.g., trajectory fragmentation and identity switches).”
Multiple Object Tracking Using K-Shortest Paths Optimization (2011, unknown): “For example, Perera et al. [2] rely on Kalman filtering to obtain basic tracks and then try to merge and split the tracks using the Hungarian algorithm.”
Tracking Multiple Occluding People by Localizing on Multiple Scene Planes (2009, unknown): “Recently, Perera et al. [ 44 ] proposed a two-stage framework, which involves one-toone correspondence followed by a split and merge analysis,”
High-throughput Ethomics in Large Groups of Drosophila (2009, methodology): “The best overall assignment was computed using the Hungarian method for minimum-weight perfect bipartite matchin”
An online learned CRF model for multi-target tracking (2012, methodology): “Due to great improvements on object detection, association based tracking approaches have been proposed [11, 17, 2, 12, 18].”
Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera (2011, result): “In contrast to data association based tracking approaches, which link detection responses to trajectories by global optimization based on position, size and appearance similarity [1], [3], [18], [26], [30], [36], [48], the combination of object detectors and particle filtering results in algorithms ”
StrongSORT: Make DeepSORT Great Again (2022, unknown): “Linear interpolation is widely used to ﬁll the gaps in recovered trajectories for missing detections [36, 22, 33, 37, 63, 11].”
Multiple object tracking: A literature review (2014, methodology): “Then either a greedy bipartite assignment algorithm [32], [111], [123] or the optimal Hungarian algorithm [31], [39], [58], [66], [124] are employed to determine the matching between nodes in the two sets.”
Robust tracking-by-detection using a detector confidence particle filter (2009, methodology): “However, many methods rely on global optimization to construct consistent trajectories from detections [1, 3, 12, 16, 21], which precludes their use in an online scenario.”
Object Detection and Tracking for Autonomous Navigation in Dynamic Environments (2010, background): “The combinatorial explosion can be limited by linking detections to ‘‘tracklets’’ where the data association is unambiguous, and optimizing data association over those tracklets (Kaucic et al. 2005; Nillius et al. 2006; Perera et al. 2006; Yan et al. 2006; Li et al. 2009).”
Global data association for multi-object tracking using network flows (2008, methodology): “People that are too small in the images(less than 24 pixel in width) or partially out of the scene are not counted in the evaluation.”
Robust Object Tracking by Hierarchical Association of Detection Responses (2008, background): “[6,7,8], an affinity score between detection responses (or tracklets) is computed once and fixed for all later processing.”
Multi-target tracking by on-line learned discriminative appearance models (2010, background): “There are two main streams in detection-based tracking methods: one considers only past and current frames to make association decisions [7] [8] [20] [25] [27]; the other takes information from future frames also [3] [5] [15] [16] [18] [21] [26] [28].”